UTL_HTTP Appendix A

Appendix A

UTL_HTTP

In this section, we’ll look at UTL _ HTTP and what it can be used for. Additionally, I would like to introduce a new and improved UTL _ HTTP built on the SocketType implemented in the UTL _ TCP section. It gives performance comparable to the native UTL _ HTTP but provides many more features.

The UTL _ HTTP package supplied with the database is relatively simplistic – it has two versions:

❑ UTL_HTTP.REQUEST

: returns up to the first 2,000 bytes of the content of a URL as a function return value.

❑ UTL_HTTP.REQUEST_PIECES

: returns a PL/SQL table of VARCHAR2(2000) elements. If you concatenated all of the pieces together – you would have the content of the page.

The UTL _ HTTP package is missing the following functionality however:

❑ You cannot inspect the HTTP headers. This makes error reporting impossible. You cannot tell the difference between a Not Found and Unauthorized for example.

❑

❑

You cannot POST information to a web server that requires POST ing data. You can only use the GET syntax. Additionally HEAD is not supported in the protocol.

You cannot retrieve binary information using UTL _ HTTP .

❑

❑

❑

❑

The request pieces API is non intuitive, the use of CLOB s and BLOB s to return the data as a

‘stream’ would be much more intuitive (and give us access to binary data).

It does not support cookies.

It does not support basic authentication.

It has no methods for URL encoding data.

1208

UTL_HTTP

One thing that UTL _ HTTP does support that we will not in our re-implementation is SSL. Using the

Oracle Wallet manager, it is possible to perform a HTTPS request (HTTPS is using SSL with HTTP).

We will demonstrate using UTL _ HTTP over SSL, but will not implement it in our own HTTP _ PKG . Due to its size, the source code of the HTTP _ PKG body will be omitted from this chapter – it is available in its entirety on the Apress website at http://www.apress.com

.

UTL_HTTP Functionality

We will look at the UTL _ HTTP functionality first as we will support its syntax on our own HTTP _ PKG .

The simplest form of UTL _ HTTP is as follows. In this example, myserver is the name I have given to the web server. Of course, you should try this example using a web server to which you have access: ops$tkyte@DEV816> select utl _ http.request( ‘http://myserver/’ ) from dual;

UTL _ HTTP.REQUEST(‘HTTP://MYSERVER/’)

-----------------------------------------------------------

<HTML>

<HEAD>

<TITLE>Oracle Service Industries</TITLE>

</HEAD>

<FRAMESET COLS=“130,*” border=0>

<FRAME SRC=“navtest.html” NAME=“sidebar” frameborder=0>

<FRAME SRC=“folder _ home.html” NAME=“body” frameborder=“0” marginheight=“0” marginwidth=“0”>

</FRAMESET>

</BODY>

</HTML>

I can simply run UTL _ HTTP .

REQUEST and send it a URL. UTL _ HTTP will connect to that web server and

GET the requested page and then return the first 2,000 characters of it. As mentioned above, don’t try to use the URL I have given above, it is for my web server inside of Oracle. You won’t be able to get to it, the request will time out with an error message.

Most networks today are protected by firewalls. If the page I wanted was only available via a firewall proxy server; I can request that as well. A discussion of firewalls and proxy servers is beyond the scope of this book. However, if you know the hostname of your proxy server, you can retrieve a page from the internet via this method: ops$tkyte@DEV816> select utl _ http.request( ‘http://www.yahoo.com’, ‘www-proxy’) from dual;

UTL _ HTTP.REQUEST(‘HTTP://WWW.YAHOO.COM’,’WWW-PROXY’)

-------------------------------------------------------------------

<html><head><title>Yahoo!</title><base href=http://www.yahoo.com/><meta httpequiv=“PICS-Label” content=‘(PICS-1.1 http://www.rsac.org/ratingsv01.html” l gen true for “http://www.yahoo.com” r (n 0 s 0 v 0 l 0))’></head><body><center><form action=http://search.yahoo.com/bin/search><map name=m><area coords=“0,0,52,52” href=r/a1><area coords=“53,0,121,52” href=r/p1><area coords=“122,0,191,52” href=r

The second parameter to UTL _ HTTP .

REQUEST and REQUEST _ PIECES is the name of a proxy server. If your proxy server is not running on the standard port 80, we can add the port as follows (In the code this is myserver on port 8000):

1209

Appendix A ops$tkyte@DEV816> select utl _ http.request( ‘http://www.yahoo.com’,

2 ‘myserver:8000’ ) from dual

3 /

UTL _ HTTP.REQUEST(‘HTTP://WWW.YAHOO.COM’,’MYSERVER:8000’)

-------------------------------------------------------

<html><head><title>Yahoo!</title><base href=http://www.yahoo.com/

So, by simply adding the :8000 to the proxy server name, we are able to connect to that proxy server.

Now, let’s look at the REQUEST _ PIECES interface: ops$tkyte@DEV816> declare

2 pieces utl _ http.html

_ pieces;

3 n number default 0;

4 l _ start number default dbms _ utility.get

_ time;

5 begin

6 pieces :=

7 utl _ http.request

_ pieces( url => ‘http://www.oracle.com/’,

8 max _ pieces => 99999,

9 proxy => ‘www-proxy’ );

10 for i in 1 .. pieces.count

11 loop

12 loop

13 exit when pieces(i) is null;

14 dbms _ output.put

_ line( substr(pieces(i),1,255) );

15 pieces(i) := substr( pieces(i), 256 );

16 end loop;

17 end loop;

18 end;

19 /

<head>

<title>Oracle Corporation</title>

<meta http-equiv=“Content-Type” content=“text/html; charset=iso-8859-1”>

<meta name=“description” content=“Oracle Corporation provides the software that powers the Internet. For more information about Oracle, pleas e call 650/506-7000.”>

The request pieces API cannot be called from SQL since it does not return a SQL type but rather a

PL/SQL table type. So, REQUEST _ PIECES is only useful inside of a PL/SQL block itself. In the above, we are requesting the web page http://www.oracle.com/ and we are requesting the first 99,999 chunks, which are each of the size 2,000 bytes. We are using the proxy server www-proxy . We must tell

REQUEST _ PIECES how many 2,000 byte chunks we are willing to accept, typically I set this to a very large number as I want the entire page back. If the information that you want from the page is always in the first 5000 bytes, you could request just 3 chunks to get it.

Adding SSL to UTL_HTTP

UTL _ HTTP also supports using SSL (Secure Sockets Layer). If you are not familiar with SSL and what it is used for, you can find a brief description at http://www.rsasecurity.com/rsalabs/faq/5-1-2.html

. Both

1210

UTL_HTTP the REQUEST and REQUEST _ PIECES functions in UTL _ HTTP support the retrieval of URLs that are protected by SSL. However, the available documentation on doing so can be described as sparse at best.

SSL support is provided by using the last two parameters in the UTL _ HTTP.REQUEST

and

UTL _ HTTP.REQUEST

_ PIECES procedures. These parameters are the WALLET _ PATH and

WALLET _ PASSWORD .

Oracle uses the wallet as a metaphor for how a person stores their security credentials; just like you would keep your driver’s license and credit cards in your wallet for identification purposes, the Oracle wallet stores the credentials needed by the SSL protocol. The WALLET _ PATH is the directory where your wallet is stored on the database server machine. This wallet is password protected to prevent someone from using your credentials. This is the purpose of the WALLET _ PASSWORD parameter, it is used to access the wallet. The password prevents people from copying the wallet directory and trying to impersonate you, as they will be unable to open and access the wallet. This is analogous to using a PIN for using an ATM machine. If someone steals your bank card, they need to have your PIN to get to your accounts.

The wallet, or the concept of a wallet is used not only by the Oracle database, but also in Web browsers.

The important aspect is that when you connect to a site, e.g., http://www.amazon.com

, how do you know that it is really Amazon.com? You have to get their certificate, which is digitally signed by someone. That someone is called a Certificate Authority or CA. How does my browser or database know to trust the CA that signed that certificate? For example, I could create a certificate for

Amazon.com and sign it from hackerAttackers.com. My browser and database should not accept this certificate even though it is a legitimate X.509 certificate.

The answer to this trust issue is the wallet stores a set of trusted certificates. A trusted certificate is a certificate from a CA that you trust. The Oracle wallet comes with some common trusted certificates.

You also have the ability to add certificates as necessary. Your browser does the same thing. If you ever connect to a site where your browser does not have the CA in its wallet, you will get a pop-up window that notifies you of this as well as a wizard that allows you to proceed or abort your connection.

Let’s see some examples of how to use SSL. First, we need to create a new wallet. You can invoke the

OWM (Oracle Wallet Manager) program on UNIX, or launch it from the Windows START menu on

Windows(it is in ORACLE HOME|NETWORK ADMINISTRATION). The screen you receive to do this will look like this:

1211

Appendix A

All you need to do is click on the NEW icon (the green ‘cube’) located on the left hand side of the display. It will prompt you for a password for this wallet, you are making up the password at this point so enter whatever password you would like. You may get a warning about a directory not existing, if you do, you should simply ignore it. It is the expected behavior if you have never created a wallet before. OWM at will then ask you to create a certificate request: you do not need to this. The certificate request is so you can get a certificate for yourself. This would be used in SSL v.3 where the server needs the identification of the client. Most Web sites do not authenticate users via certificates, but rather by using a username and password. This is because an e -

commerce site doesn’t care who is buying from them as long as they get the money. But you do care that you are sending the money (and credit card information) to the correct entity, so we use SSL v.2 to identify the server for example, Amazon.com, and to provide all the encryption of data. So, click NO in response to this and save the wallet by clicking on the SAVE WALLET (the yellow floppy disk) icon and we are ready to go.

Let’s go to Amazon.com first. Amazon’s certificate was signed by Secure Server Certificate Authority,

RSA Data Security, Inc. This is one of the defaults in the Oracle wallet. tkyte@TKYTE816> declare

2 l _ output long;

3

4 l _ url varchar2(255) default

5 ‘https://www.amazon.com/exec/obidos/flex-sign-in/’;

6

7 l _ wallet _ path varchar2(255) default

8 ‘file:C:\Documents and Settings\Thomas Kyte\ORACLE\WALLETS’;

9

10

11 begin

12 l _ output := utl _ http.request

13 ( url => l _ url,

14 proxy => ‘www-proxy.us.oracle.com’,

15 wallet _ path => l _ wallet _ path,

16 wallet _ password => ‘oracle’

17 );


_ line(trim(substr(l _ output,1,255)));

19 end;

20 /

<html>

<head>

<title>Amazon.com Error Page</title>

</head>

<body bgcolor=“#FFFFFF” link=“#003399” alink=“#FF9933” vlink=“#996633” text=“#000000”>

1212

UTL_HTTP

<a name=“top”></a>

<table border=0 width=100% cellspacing=0 cellpadding=0>

<tr

PL/SQL procedure successfully completed.

Don’t worry about getting this error page; this is accurate. The reason for receiving this error page is that there is no session information being passed. We are just testing that the connection worked here; we retrieved an SSL protected document.

Let’s try another site. How about E*Trade? tkyte@TKYTE816> declare

2 l _ output long;

3


5 ‘https://trading.etrade.com/’;

6



9

10

11 begin


13 ( url => l _ url,




17 );



19 end;

20 / declare

*

ERROR at line 1:

ORA-06510: PL/SQL: unhandled user-defined exception

ORA-06512: at “SYS.UTL

_ HTTP”, line 174

ORA-06512: at line 12

That apparently does not work. E*Trade has a certificate signed by. www.verisign com/CPS Incorp.by

Ref , which is not a default trusted certificate. In order to access this page, we’ll have to add that certificate to our Oracle wallet – assuming of course that we trust Verisign! Here is the trick. Go to the site ( https://trading.etrade.com

). Double-click the PADLOCK icon on the bottom right corner of the window (in Microsoft Internet Explorer). This will pop-up a window that looks similar to this one:

1213

Appendix A

Select the Certification Path tab on the top of this screen. This lists the certificate you are viewing here it is the one for E*Trade (trading.etrade.com), as well as who issued the certificate. We need to add the person who signed the certificate (the issuer) to our trusted certificates in the Oracle wallet. The issuer is www.verisign.com/CPS Incorp.by Ref. LIABILITY LTD as depicted by the tree-like hierarchy.

Click the View Certificate button while www.verisign.com/CPS Incorp. by Ref . is highlighted. This shows information for the issuer’s certificate. Click the Details tab and you should see:

1214

UTL_HTTP

Now we need to Click the Copy to File button. Save the file locally as a Base-64 encoded X.509 (CER) file. The following screen shows the selection you should make, you can name the file anything you choose and save it anywhere. We’ll be importing it in a moment, just remember where you save it to:

Now we can import this into our Oracle Wallet. Open the wallet in OWM and right click on the

Trusted Certificates – this will present you with a popup menu that has Import Trusted Certificate :

1215

Appendix A

You will select that option and in the next dialog that comes up, choose Select a file that contains the certificate

Use the standard ‘file open’ dialog that comes up to choose the certificate you just saved. Your screen should now look something like this:

Now, save the wallet using clicking on the SAVE WALLET (the yellow floppy disk) icon and let’s try our example again: tkyte@TKYTE816> declare

2 l _ output long;

3


5 ‘https://trading.etrade.com/cgi-bin/gx.cgi/AppLogic%2bHome’;

6



9

10

11 begin


13 ( url => l _ url,




17 );



19 end;

20 /

1216

UTL_HTTP

<HTML>

<HEAD>

<META http-equiv=“Content-Type” content=“text/html; charset=ISO-8859-1”>

<TITLE>E*TRADE</TITLE>

<SCRIPT LANGUAGE=“Javascript”

TYPE=“text/javascript”>

<!--

function mac _ comment(){

var agt=navigator.userAgent.toLowerCase();

var is _ mac

PL/SQL procedure successfully completed.

This time we are successful. Now we know how to use and extend the Oracle wallet to do secure

HTTPS.

Really Using UTL_HTTP

Well, besides getting the content of a web page which is extremely useful – what else can we do with

UTL _ HTTP ? Well, a common use for it is an easy way to make it so that PL/SQL can run a program – sort of like a HOST command. Since almost every web server can run cgi-bin programs and UTL _ HTTP can send URLs, we can in effect have PL/SQL execute host commands by configuring the commands we wanted to execute as cgi-bin programs under the webserver.

What I like to do in this case is setup a web server running on IP address 127.0.0.1

– which is the

TCP/IP loop back. This IP address is only accessible if you are physically logged onto the machine the web server is running on. In that fashion, I can set up cgi-bin programs for my PL/SQL programs to run that no one else can – unless they break in to my server machine in which case I have much larger problems to deal with.

One use I’ve made of this facility in the past is to send e-mail from PL/SQL. Let’s say you have the

Oracle 8i database without Java in it. Without Java you cannot use either UTL _ TCP or UTL _ SMTP – they both rely on the Java option in the database. So, without UTL _ SMTP and/or UTL _ TCP , how could we send an email? With UTL _ HTTP and UTL _ FILE – I can set up a cgi-bin program that receives a single input in the QUERY _ STRING environment variable. The single input would be a file name. We will use

/usr/lib/sendmail to send that file (on Windows I could use the public domain utility ‘blat’ to send mail, available from http://www.interlog.com/~tcharron/blat.html

). Once I have that set up, I can simply run my host command via:

…

results := utl _ http.request(

‘http://127.0.0.1/cgi-bin/smail?filename.text’ );

…

The cgi-bin program I set up, smail , would return a ‘web page’ that indicated success or failure, I would look at the result’s variable to see if the mail was sent successfully or not. The full fledged implementation of this on Unix could be:

1217

Appendix A scott@ORA8I.WORLD> create sequence sm _ seq

2 /

Sequence created. scott@ORA8I.WORLD> create or replace procedure sm( p _ to in varchar2,

2 p _ from in varchar2,

3 p _ subject in varchar2,

4 p _ body in varchar2 )

5 is

6 l _ output utl _ file.file

_ type;

7 l _ filename varchar2(255);

8 l _ request varchar2(2000);

9 begin

10 select ‘m’ || sm _ seq.nextval || ‘.EMAIL.’ || p _ to

11 into l _ filename

12 from dual;

13

14 l _ output := utl _ file.fopen

15 ( ‘/tmp’, l _ filename, ‘w’, 32000 );

16

17 utl _ file.put

_ line( l _ output, ‘From: ‘ || p _ from );

18 utl _ file.put

_ line( l _ output, ‘Subject: ‘ || p _ subject );

19 utl _ file.new

_ line( l _ output );

20 utl _ file.put

_ line( l _ output, p _ body );

21 utl _ file.new

_ line( l _ output );

22 utl _ file.put

_ line( l _ output, ‘.’ );

23

24 utl _ file.fclose( l _ output );

25

26 l _ request := utl _ http.request

27 ( ‘http://127.0.0.1/cgi-bin/smail?’ || l _ filename );

28


_ line( l _ request );

30 end sm;

31 /

Procedure created.

You should refer to the section on UTL _ SMTP to understand why I formatted the email as I did here with the From: and To : header records. In this routine we are using a sequence to generate a unique filename. We encode the recipient in the name of the file itself. Then, we write the email to an OS file.

Lastly, we use UTL _ HTTP to run our host command and pass it the name of the file. We simply print out the result of this in this test case – we would really be inspecting the value of l _ result to ensure the email was sent successfully.

The simple cgi-bin program could be:

#!/bin/sh echo “Content-type: text/plain” echo ““ echo $QUERY _ STRING to=ècho $QUERY _ STRING|sed ‘s/.*EMAIL\.//’` echo $to

(/usr/lib/sendmail $to < /tmp/$QUERY _ STRING) 1>> /tmp/$$.log 2>&1 cat /tmp/$$.log rm /tmp/$$.log rm /tmp/$QUERY _ STRING

1218

UTL_HTTP

The shell script starts by printing out the HTTP headers that we need in order to return a document; that is what the first two echoes are doing. Then, we are simply printing out the value of the

QUERY _ STRING (this is where the web server puts our inputs – the portion after the ?

in the URL). We then extract the email address from the QUERY _ STRING by using the sed (Stream EDitor) to remove everything in front of the word EMAIL/ in the filename. We then run sendmail in a subshell so as to be able to capture both its stdout and stderr output streams. I cat (type to stdout ) the contents of the log we captured from sendmail. In this case I wanted both stdout and stderr to be returned as the contents of the web page, so the PL/SQL code could get any error messages and such. The easiest way to do that was to redirect both output streams to a temporary file and then type that file to stdout . We then clean up our temp files and return.

I can now test this via: scott@ORA8I.WORLD> begin

2 sm( ‘tkyte@us.oracle.com’,

3 ‘tkyte@us.oracle.com’,

4 ‘testing’,

5 ‘hello world!’ );

6 end;

7 / m1.EMAIL.tkyte@us.oracle.com tkyte@us.oracle.com

This shows us the QUERY _ STRING and the To: environment variables, and since there is no other test

(no error messages) we know the e-mail was sent.

So, this small example shows how UTL _ HTTP can be used indirectly for something it was not really designed for; to allow PL/SQL to run HOST commands. You would set up a special purpose web server on IP Address 127.0.0.1, configure a cgi-bin type directory within it and place the commands you want

PL/SQL developers to run in there. Then you have a secure way to allow PL/SQL to run the equivalent of SQL*PLUS’s host command.

A Better UTL_HTTP

Given that we have the SocketType class developed in the UTL _ TCP section (or just access to UTL _ TCP alone) and knowledge of the HTTP protocol, we can make an improved UTL _ HTTP package. We will call our implementation HTTP _ PKG . It will support the “old fashioned” UTL _ HTTP interface of REQUEST and REQUEST _ PIECES but will also add support for:

❑ Getting the HTTP headers back with each request – These headers contain useful information such as the status of the request (e.g. 200 OK , 404 Not Found , and so on), the name of the server that executed the URL, the content type of the returned content, cookies and so on.

❑ Getting the content back as either a CLOB or a BLOB – This allows your PL/SQL to retrieve a large PDF file to be inserted into a table indexed by interMedia as well as retrieve plain text from another page or just to access any binary data returned by a web server.

❑

❑

Perform a HEAD of a document – This is useful to check and see if the document you retrieved last week has been updated for example.

URL-encoded strings – for example, if you have a space or a tilde (~) in a URL request, they must be ‘escaped’. This function escapes all characters that need to be.

1219

Appendix A

❑

❑ POST data instead of just GET – The GET protocol has limits that vary by web server on the size of the request. Typically, a URL should not exceed 1 to 2 KB in length. If it does, you should POST the data. POST ed data can be of unlimited size.

We can implement all of this in PL/SQL using our SocketType from the UTL _ TCP section and we can do it with a fairly small amount of code. The specification for our new HTTP _ PKG package follows. We’ll look at the specification and some examples that use it. What will not be in this book is the code that implements the body of the HTTP _ PKG . This code is available and documented on the Apress website,

( http://www.apress.com

), it is about 15 printed pages long hence it is not included here. The package specification is as follows. The first two functions, REQUEST and REQUEST _ PIECES , are functionally equivalent (minus SSL support) to functions found in the UTL _ HTTP package in versions 7.3.x, 8.0.x, and 8.1.5, we even will raise the same sort of named exceptions they would: tkyte@TKYTE816> create or replace package http _ pkg

2 as

3 function request( url in varchar2,

4 proxy in varchar2 default NULL )

5 return varchar2;

6

7 type html _ pieces is table of varchar2(2000)

8 index by binary _ integer;

9

10 function request _ pieces(url in varchar2,

11 max _ pieces natural default 32767,

12 proxy in varchar2 default NULL)

13 return html _ pieces;

14

15 init _ failed exception;

16 request _ failed exception;

The next procedure is GET _ URL . It invokes the standard HTTP GET command on a web server. The inputs are:

❑ p _ url is the URL to retrieve.

❑

Sending Cookies with requests – If you are using this HTTP _ PKG to get access to a web site that uses a cookie based authentication scheme, this is crucial. You would have to GET the login page, sending the username and password in the GET request. You would then look at the HTTP headers returned by that page and extract their cookie. This cookie value is what you need to send on all subsequent requests to prove who you are.

❑

❑ p _ proxy is the name:<port> of the proxy server to use. Null indicates you need not use a proxy server. Examples: p _ proxy => ‘www-proxy’ or p _ proxy => ‘www-proxy:80’ . p _ status is returned to you. it will be the HTTP status code returned by the web server. 200 indicates normal, successful completion. 401 indicates unauthorized, and so on. p _ status _ txt is returned to you. It contains the full text of the HTTP status record. For example it might contain: HTTP/1.0 200 OK .

1220

UTL_HTTP

!

! p _ httpHeaders may be set by you and upon return will contain the http headers from the requested

URL. On input, any values you have set will be transmitted to the web server as part of the request. On output, the headers generated by the web server will be returned to you. You can use this to set and send cookies or basic authentication or any other http header record you wish. p _ content is a temporary CLOB or BLOB (depending on which overloaded procedure you call) that will be allocated for you in this package (you need not allocate it). It is a session temporary LOB . You may use dbms _ lob.freetemporary

to deallocate it whenever you want, or just let it disappear when you log out.

19 procedure get _ url( p _ url in varchar2,

20 p _ proxy in varchar2 default NULL,

21 p _ status out number,

22 p _ status _ txt out varchar2,

23 p _ httpHeaders in out CorrelatedArray,

24 p _ content in out clob );

25

26

27 procedure get _ url( p _ url in varchar2,





32 p _ content in out blob );

The next procedure is HEAD _ URL . Invokes the standard HTTP HEAD syntax on a web server. The inputs and outputs are identical to get _ url above (except no content is retrieved). This function is useful to see if a document exists, what its mime-type is, or if it has been recently changed, without actually retrieving the document itself:

34 procedure head _ url( p _ url in varchar2,




38 p _ httpHeaders out CorrelatedArray );

The next function URL encode is used when building GET parameter lists or building POST CLOB s. It is used to escape special characters in URLs (for example, a URL may not contain a whitespace, a % sign, and so on). Given input such as Hello World , then urlencode will return Hello%20World ’

40 function urlencode( p _ str in varchar2 ) return varchar2;

Procedure Add _ A _ Cookie allows you to easily set a cookie value to be sent to a web server. You need only know the name and value of the cookie. The formatting of the HTTP header record is performed by this routine. The p _ httpHeaders variable you send in/out of this routine would be sent in/out of the <Get|Head|Post> _ url routines:

42 procedure add _ a _ cookie

43 ( p _ name in varchar2,

44 p _ value in varchar2,

45 p _ httpHeaders in out CorrelatedArray );

1221

Appendix A

The next procedure Set _ Basic _ Auth allows you to enter a username/password to access a protected page. The formatting of the HTTP header record is performed by this routine. The p _ httpHeaders variable you send in/out of this routine would be sent in/out of the <Get|Head|Post> _ url routines as well:

47 procedure set _ basic _ auth

48 ( p _ username in varchar2,

49 p _ password in varchar2,

50 p _ httpHeaders in out CorrelatedArray );

The procedure set _ post _ parameter is used when retrieving a URL that needs a large (greater than

2,000 or so bytes) set of inputs. It is recommended that the POST method be used for large requests.

This routine allows you to add parameter after parameter to a POST request. This post request is built into a CLOB which you supply:

52 procedure set _ post _ parameter

53 ( p _ name in varchar2,

54 p _ value in varchar2,

55 p _ post _ data in out clob,

56 p _ urlencode in boolean default FALSE );

The next two routines are identical to GET _ URL above with the addition of the p _ post _ data input p _ post _ data is a CLOB built by repeated calls to set _ post _ parameter above. The remaining inputs/outputs are defined the same as they were for GET _ URL :

58 procedure post _ url

59 (p _ url in varchar2,

60 p _ post _ data in clob,





65 p _ content in out clob );

66

67 procedure post _ url

68 (p _ url in varchar2,

69 p _ post _ data in clob,





74 p _ content in out blob );

75

76

77 end;

78 /

Package created.

So, the specification of the package is done; the procedures defined within it are rather straightforward.

I can do things like GET _ URL to get a URL. This will use the HTTP GET syntax to retrieve the contents of a web page into a temporary BLOB or CLOB . I can HEAD _ URL to get the headers for a URL. Using this

I could look at the mime type for example to decide if I wanted to use a CLOB ( text/html ) to get the

1222

UTL_HTTP

URL or a BLOB ( image/gif ). I can even POST _ URL to post large amounts of data to a URL. There are the other helper functions to set cookies in the header, to base 64 encode the username and password for basic authentication and so on.

Now, assuming you have downloaded the implementation of HTTP _ PKG (the package body consists of about 500 lines of PL/SQL code) we are ready to try it out. We’ll introduce a pair of utility routines that will be useful to test with first. Print _ clob below simply prints out the entire contents of a CLOB using

DBMS _ OUTPUT . Print _ Headers does the same for the HTTP headers we retrieve above in our

CorrelatedArray type (an object type that is part of the HTTP _ PKG implementation). In the following, the procedure P is the P procedure I introduced in the DBMS _ OUTPUT section to print long lines: ops$tkyte@DEV816> create or replace procedure print _ clob( p _ clob in clob )

2 as

3 l _ offset number default 1;

4 begin

5 loop

6 exit when l _ offset > dbms _ lob.getlength(p _ clob);

7 dbms _ output.put

_ line( dbms _ lob.substr( p _ clob, 255, l _ offset ) );

8 l _ offset := l _ offset + 255;

9 end loop;

10 end;

11 / ops$tkyte@DEV816> create or replace

2 procedure print _ headers( p _ httpHeaders correlatedArray )

3 as

4 begin

5 for i in 1 .. p _ httpHeaders.vals.count loop

6 p( initcap( p _ httpHeaders.vals(i).name ) || ‘: ‘ ||

7 p _ httpHeaders.vals(i).value );

8 end loop;

9 p( chr(9) );

10 end;

11 /

Now onto the test: ops$tkyte@DEV816> begin

2 p( http _ pkg.request( ‘http://myserver/’ ) );

3 end;

4 /

<HTML>

<HEAD>

<TITLE>Oracle Service Industries</TITLE>

</HEAD>

<FRAMESET COLS=“130,*” border=0>

<FRAME SRC=“navtest.html” NAME=“sidebar” frameborder=0>

<FRAME SRC=“folder _ home.html”

NAME=“body” frameborder=“0” marginheight=“0” marginwidth=“0”>

</FRAMESET>

</BODY>

</HTML>

1223

Appendix A ops$tkyte@DEV816> declare

2 pieces http _ pkg.html

_ pieces;

3 begin

4 pieces :=

5 http _ pkg.request

_ pieces( ‘http://www.oracle.com’,

6 proxy=>‘www-proxy1’);

7

8 for i in 1 .. pieces.count loop

9 p( pieces(i) );

10 end loop;

11 end;

12 /

<head>

<title>Oracle Corporation</title>

<meta http-equiv=“Content-Type” content=“text/html;

…

The above two routines simply show that the UTL _ HTTP methods of REQUEST and REQUEST _ PIECES function as expected with our new package. Their functionality is identical. Now we will invoke our

URLENCODE function that translates ‘bad characters into escape sequences in URLs and POST data: ops$tkyte@DEV816> select

2 http _ pkg.urlencode( ‘A>C%{hello}\fadfasdfads~`[abc]:=$+’’”’ )

3 from dual;

HTTP _ PKG.URLENCODE(‘A>C%{HELLO}\FADFASDFADS~`[ABC]:=$+’’”’)

---------------------------------------------------------------

A%3EC%25%7Bhello%7D%5Cfadfasdfads%7E%60%5Babc%5D%3A%3D%24%2B%27%22

That shows that characters like > and % are escaped into %3E and %25 respectively and other sequences such as the word hello are not escaped. This allows us to use any of these special characters in our

HTTP requests safely.

Now we will see the first of the new HTTP URL procedures. This procedure call will return Yahoo’s home page via a proxy server, www-proxy1 , (you’ll need to replace this with your own proxy server of course). Additionally, we get to see the HTTP status returned – 200 indicates success. We also see the

HTTP headers Yahoo returned to us. The mime-type will always be in there and that tells us what type of content we can expect. Lastly, the content is returned and printed out: ops$tkyte@DEV816> declare

2 l _ httpHeaders correlatedArray;

3 l _ status number;

4 l _ status _ txt varchar2(255);

5 l _ content clob;

6 begin

7 http _ pkg.get

_ url( ‘http://www.yahoo.com/’,

8 ‘www-proxy1’,

9 l _ status,

10 l _ status _ txt,

11 l _ httpHeaders,

12 l _ content );

13

14 p( ‘The status was ‘ || l _ status );

1224

UTL_HTTP

15 p( ‘The status text was ‘ || l _ status _ txt );

16 print _ headers( l _ httpHeaders );

17 print _ clob( l _ content );

18 end;

19 /

The status was 200

The status text was HTTP/1.0 200 OK

Date: Fri, 02 Feb 2001 19:13:26 GMT

Connection: close

Content-Type: text/html

<html><head><title>Yahoo!</title><base href=http://www.yahoo.com/><meta httpequiv=“PICS-Label”

Next, we will try the HEAD request against the home page of a sample site, let’s call it, Sample.com, and see what we can discover: ops$tkyte@DEV816> declare




5 begin

6 http _ pkg.head

_ url( ‘http://www.sample.com/’,

7 ‘www-proxy1’,

8 l _ status,

9 l _ status _ txt,

10 l _ httpHeaders );

11




15 end;

16 /

The status was 200


Server: Microsoft-IIS/5.0

Date: Fri, 02 Feb 2001 19:13:26 GMT

Connection: Keep-Alive

Content-Length: 1270


Set-Cookie: ASPSESSIONIDQQQGGNQU=PNMNCIBACGKFLHGKLLBPEPMD; path=/

Cache-Control: private

From the headers, it is obvious that Sample.com is running Windows with Microsoft IIS. Further, they are using ASP’s as indicated by the cookie they sent back to us. If we were to have retrieved that page, it would have had 1,270 bytes of content.

Now we would like to see how cookies might work. Here I am using a standard procedure that is used with OAS (the Oracle Application Server) and iAS (Oracle’s Internet Application Server); the cookiejar sample that shows how to use cookies in a PL/SQL web procedure. The routine cookiejar looks at the cookie value and if set, increments it by one and returns it to the client. We’ll see what how that would work using our package. We are going to send the value 55 to the server and we are expecting it to send us 56 back:

1225

Appendix A ops$tkyte@DEV816> declare




5 l _ content clob;

6 begin

7 http _ pkg.add

_ a _ cookie( ‘COUNT’, 55, l _ httpHeaders );

8 http _ pkg.get

_ url

9 ( ‘http://myserver.acme.com/wa/webdemo/owa/cookiejar’,

10 null,

11 l _ status,


13 l _ httpHeaders,

14 l _ content );

15





20 end;

21 /

The status was 200



Date: Fri, 02 Feb 2001 19:14:48 GMT

Allow: GET, HEAD

Server: Oracle _ Web _ listener2.1/1.20in2

Set-Cookie: COUNT= 56 ; expires=Saturday, 03-Feb-2001 22:14:48 GMT

<HTML>

<HEAD>

<TITLE>C is for Cookie</TITLE>

</HEAD>

<BODY>

<HR>

<IMG SRC=“/ows-img/ows.gif”>

<H1>C is for Cookie</H1>

<HR>

You have visited this page <STRONG> 56 </STRONG> times in the last 24 hours.

…

As you can see, the cookie value of 55 was transmitted and the server incremented it to 56. It then sent us back the modified value along with an expiration date.

Next, we would like to see how to access a page that requires a username and password. This is done via the following:

1226

UTL_HTTP ops$tkyte@DEV816> declare




5 l _ content clob;

6 begin

7 http _ pkg.set

_ basic _ auth( ‘tkyte’, ‘tiger’, l _ httpheaders );

8 http _ pkg.get

_ url

9 ( ‘http://myserver.acme.com:80/wa/intranets/owa/print _ user’,

10 null,

11 l _ status,


13 l _ httpHeaders,

14 l _ content );

15




19 print _ clob(l _ content);

20 end;

21 /

The status was 200



Date: Fri, 02 Feb 2001 19:49:17 GMT

Allow: GET, HEAD

Server: Oracle _ Web _ listener2.1/1.20in2 remote user = tkyte

Here, I just set up a DAD (Database Access Descriptor) that did not store the username/password with the DAD. This means the web server is expecting the request to contain the username/password to use.

Here I passed my credentials to a routine that simply printed out the REMOTE _ USER cgi-environment variable in PL/SQL (the name of the remotely connected user).

Lastly, we would like to demonstrate the POST ’ing of data. Here I am using a URL from Yahoo again.

Yahoo makes it easy to get stock quotes in a spreadsheet format. Since the list of stock symbols you might be interested in could get quite large, I would suggest POST ing this data. Here is an example that gets a couple of stock quotes from Yahoo using HTTP. The data will be returned in CSV (Comma

Separated Values) for easy parsing and loading into a table for example: ops$tkyte@DEV816> declare




5 l _ content clob;

6 l _ post clob;

7 begin

8 http _ pkg.set

_ post _ parameter( ‘symbols’,’orcl ÎXID ^DJI ^SPC’,

9 l _ post, TRUE );

10 http _ pkg.set

_ post _ parameter( ‘format’, ‘sl1d1t1c1ohgv’,

11 l _ post, TRUE );

12 http _ pkg.set

_ post _ parameter( ‘ext’, ‘.csv’,

1227

Appendix A

13 l _ post, TRUE );

14 http _ pkg.post

_ url( ‘http://quote.yahoo.com/download/quotes.csv’,

15 l _ post,

16 ‘www-proxy’,

17 l _ status,


19 l _ httpHeaders,

20 l _ content );

21





26 end;

27 /

The status was 200


Date: Fri, 02 Feb 2001 19:49:18 GMT

Cache-Control: private

Connection: close

Content-Type: application/octet-stream

“ORCL”,28.1875,”2/2/2001”,”2:34PM”,-1.875,29.9375,30.0625,28.0625,26479100

“ÎXID”,1620.60,”2/2/2001”,”2:49PM”,-45.21,1664.55,1667.46,1620.40,N/A

“^DJI”,10899.33,”2/2/2001”,”2:49PM”,-84.30,10982.71,11022.78,10888.01,N/A

“^SPC”,1355.17,”2/2/2001”,”2:49PM”,-18.30,1373.53,1376.16,1354.21,N/A

Summary

In this section, we have seen how to use the built-in UTL _ HTTP package. We have seen how through a little creative thinking, we can use UTL _ HTTP not only to grab data from the web, but to enable PL/SQL to have the equivalent of a HOST command. With just a couple of lines of code, we made it possible that any release of Oracle from 7.3 on up can easily send mail using UTL _ FILE and UTL _ HTTP .

We then investigated how to use UTL _ HTTP over SSL – a concept not well documented in the Oracle

Supplied Packages Guide (or any other document for that matter). We learned how to make any SSL enabled website available to our PL/SQL routines using the Oracle Wallet Manager.

Additionally, we’ve seen how we can take a good idea and make it better. In comparison to the section on DBMS _ OBFUSCATION _ TOOLKIT where we ‘wrapped’ the functionality of that package to make it easier and more flexible to use, here we totally re-implemented a package giving it additional functionality it never had. This came at a certain price, we do not support SSL in our implementation, but it is useful in many cases nonetheless.

We could easily take this a step further and user either Java or a C based External Procedure to add full support for SSL as well. There are also various third party and public domain classes and libraries out there to do just that.

1228

UTL_HTTP Appendix A

UTL_HTTP

UTL_HTTP Functionality

Adding SSL to UTL_HTTP

Really Using UTL_HTTP

A Better UTL_HTTP

Summary

Related documents

Products

Support

UTL_HTTP Appendix A

UTL_HTTP

UTL_HTTP Functionality

Adding SSL to UTL_HTTP

Really Using UTL_HTTP

A Better UTL_HTTP

Summary

Related documents

Add this document to collection(s)

Add this document to saved

Suggest us how to improve StudyLib