Tag: ADB

  • Example: an ORDS API that returns a user-configured JSON object

    Example: an ORDS API that returns a user-configured JSON object

    This actually came about from a Support Request. We were troubleshooting a JWT issue, that had ORDS in the mix. Ultimately, this is looking more and more like a networking issue (maybe something to do with routing rules between the load balancer and the backend server).

    But in the course of our troubleshooting, we wanted to rule out some variables. One of those variables was the JWK URL we were using for the ORDS_SECURITY.CREATE_JWT_PROFILE procedure. So we had this idea to kind of fool ORDS into thinking it was visiting the IDCS (the predecessor to Oracle IAM) JWK URL when, in fact, it was just visiting another ORDS endpoint.

    If you’ve seen my JWT + ORDS tutorials (one for OCI IAM, another for Microsoft Azure), then you know when you create an ORDS JWT Profile, it will look something like this:

    BEGIN
      ORDS_SECURITY.CREATE_JWT_PROFILE(
          p_issuer => 'https://identity.oraclecloud.com/',
          p_audience => 'my_primary_audience' ,
          p_jwk_url =>'https://idcs-10a10a10a10a10a10a10a10a.identity.oraclecloud.com/admin/v1/SigningCert/jwk'
      );
      COMMIT;
    END;
    /

    In the above code block, just know that p_issuer is your Identity Provider (e.g., IAM, IDCS, Microsoft Entra, etc.), p_audience is the Primary Audience (which you created when you registered your client application with your Identity Provider), and the p_jwk_url is where ORDS goes to look up the keys for decoding a JWT.

    The eventual JWT that ORDS receives would include the Issuer and Audience, too. But it would also include other values such as the Scope. That Scope name would be associated with the ORDS Privilege–the privilege you created specifically to use for protecting/associating with your ORDS Resource Module. And ORDS would use all these values (and the JWK “keys” found at that JWK URL to decode the JWT) together to:

    1. Verify you are who you say you are, and
    2. You have the proper Scope (or as ORDS interprets it…the proper privilege) to access this protected resource

    Back to the use case. So, for troubleshooting reasons, we took that JSON object located at that JWK URL and recreated it as an ORDS endpoint. Let me point out that this is cool, but what you should take away from this is that there are tons of things you can do with ORDS, and you have a lot of control over how ORDS will send back information to you, your application, or your web client.

    Here is how we structured the Handler code:

    BEGIN
        HTP.P('{"
        keys": [
            {
                "kty": "RSA",
                "x5t#S256": "Lorem ipsum dolor sit amet__pellentesque elementum",
                "e": "AQAB",
                "x5t": "xwSmccaQZDvAZPOpZPHOiQDlLgo",
                "kid": "SIGNING_KEY",
                "x5c": [
                    "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Fusce lorem dui, elementum et odio ut, sollicitudin gravida dui. Ut nec dapibus elit. Morbi sit amet blandit justo. Quisque accumsan turpis metus, vel sodales nisl porta vel. Nunc placerat, mi id eleifend pharetra, massa urna imperdiet arcu, vitae dignissim nulla ex sed felis. Maecenas auctor risus ac nisl efficitur, sit amet euismod odio finibus. Praesent lacinia nunc id ex tempor, sed tempor nisl porta. Integer semper efficitur arcu, a dictum quam vulputate tempus. Nulla congue dapibus tortor vel volutpat. Curabitur et sollicitudin purus. Mauris quis aliquam augue. Nulla ac leo tristique, ultricies mauris sit amet, ultrices quam. Praesent placerat a lectus sit amet dapibus. Donec rhoncus felis velit, sed placerat nunc pellentesque elementum==",
    "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Fusce lorem dui, elementum et odio ut, sollicitudin gravida dui. Ut nec dapibus elit. Morbi sit amet blandit justo. Quisque accumsan turpis metus, vel sodales nisl porta vel. Nunc placerat, mi id eleifend pharetra, massa urna imperdiet arcu, vitae dignissim nulla ex sed felis. Maecenas auctor risus ac nisl efficitur, sit amet euismod odio finibus. Praesent lacinia nunc id ex tempor, sed tempor nisl porta. Integer semper efficitur arcu, a dictum quam vulputate tempus. Nulla congue dapibus tortor vel volutpat. Curabitur et sollicitudin purus. Mauris quis aliquam augue. Nulla ac leo tristique, ultricies mauris sit amet, ultrices quam. Praesent placerat a lectus sit amet dapibus. Donec rhoncus felis velit, sed placerat nunc pellentesque elementum"
                ],
                "key_ops": [
                    "verify",
                    "wrapKey",
                    "encrypt"
                ],
                "alg": "RS256",
                "n": 
    "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Fusce lorem dui, elementum et odio ut, sollicitudin gravida dui. Ut nec dapibus elit. Morbi sit amet blandit justo. Quisque accumsan turpis metus, vel sodales nisl porta vel. Nunc placerat, mi id eleifend pharetra, massa urna imperdiet arcu, vitae dignissim nulla"
            }
        ]
    }');
    END;

    With that new ORDS endpoint, we (temporarily) recreated the ORDS JWT Profile with the new ORDS API as a proxy for our true JWK URL. Like this (this JWK URL is entirely fake, BTW):

    BEGIN
      ORDS_SECURITY.CREATE_JWT_PROFILE(
          p_issuer => 'https://identity.oraclecloud.com/',
          p_audience => 'my_primary_audience' ,
          p_jwk_url => 'https://abcdefg-myadb.region-01.oraclecloudapps.com/ords/admin/jwk/jwk'
      );
      COMMIT;
    END;
    /

    Hypothetically if you were to navigate to that ORDS URI, you’d see a facsimile of the actual JWK information:

    So now, we’ve temporarily removed the JWK URL from the troubleshooting equation. However, we can still access that protected endpoint since we have a valid token and keys to decode it (courtesy of the ORDS endpoint we created)!

    And this my friends, is the type of mad scientist stuff that I live for. I had to share our experience to underscore two main points:

    1. You have a lot of flexibility with these ORDS APIs, and
    2. When it comes to troubleshooting/root cause analysis, you must break things down into individual parts!

    And that’s it for today. Class dismissed 🤓

    BTW, I took some inspiration from here too. And of course, a link to the JWT section of our docs.

    If you liked this, consider sharing. And if you have any cool or clever ORDS tricks you use, comment below!

    Follow

    And don’t forget to follow, like, subscribe, share, taunt, troll, or stalk me!

  • ORDS as a JotForm webhook: An actual customer use case

    Recently, we (one of our Support engineers and I) worked with a customer who had integrated various ORDS POST APIs with a web-based form application called JotForm. JotForm offers integrations for numerous workflows, some official and others you can create independently. That’s where we come in.

    In this example, a user would enter information into an online JotForm form, and when the user hits Submit, that would trigger a Webhook to an ORDS endpoint. But in this customer’s scenario, some of the data he was expecting wasn’t arriving in his database (he was on a 21c, but I tested these in a 23ai Autonomous database—the same still applies to both). For our customer, some of the fields in his table showed NULL.

    A Webhook example

    We wanted to understand better what was happening, so I created an account in Jotform and set up a Webhook similar to his configuration. This Webhook essentially forwards the JotForm form data to a target endpoint (an ORDS POST handler in our case).

    Webhooks are weird

    Here is what I don’t like about Webhooks (perhaps this is just my ignorance on full display):

    1. What if the request doesn’t work or fails? Are reattempts automatic, or is that user information lost forever?
    2. In this case, we have no real idea what is being sent to ORDS (at least not without some creative interrogation).
    3. Without spending too much time with Webhooks, would an APEX front end not be more straightforward (i.e., same ecosystem and it is a direct POST as opposed to a Webhook that forwards a POST request to an POST endpoint…say that five times fast! More on this later…).

    Where our problem starts

    Here, we have an online form in JotForm. On the front end, I’ve set up three fields (these are all customizable in their web application):

    1. Full Name, which consists of First Name and Last Name
    2. Contact number
    3. And a “Radio” button selection 1

    The Workflow

    A user enters their information and clicks Submit. However, after inspecting the Console and Network tabs (in your Developer tools), you’ll notice two things:

    1. No POST requests
    2. No available request body (What does JotForm send to us?)

    The absence of information is unsurprising since I’m working with a third-party tool (it’s a business, man!). There are enumerable reasons why you’d obscure information. Regardless of the cause, that obfuscation complicates matters when you want to ingest the form data into your database. You can follow the debugging advice to view the PHP output sent internally to JotForm. But I don’t know PHP, and I wouldn’t even know how to map PHP to JSON or SQL datatypes, for that matter.

    ORDS Implicit Parameters to the Rescue

    Here’s where you have to get creative. To better understand what was in that POST request, I relied on ORDS’ Implicit Bind parameters to understand what data was sent to us by the Webhook. Figuring this out required some trial and error on my part. But you can start with the :content_type bind parameter to see what kind of POST it is and go from there. In my example, I kept it simple and relied on the :content_type parameter and whatever was included in the POST body (up to this point, I wasn’t sure what to expect). I named that second value incoming_payload (which maps to the “PAYLOAD” column in the JOTFORM_DATA table)

    INFO: I used the :body_json bind parameter, for the sole fact that neither the :body nor the :body_text parameters seemed to work for me. 

    Resource Handler code

    A look at the ORDS Resource Handler code:

    -- The :content_type implicit bind parameter would "map" to the CONTENT_TYPE column in the 
    -- JOTFORM_DATA table, while incoming_payload would map to the PAYLOAD column. 
    
    DECLARE
        incoming_payload CLOB;
    BEGIN
        incoming_payload := :body_json;
        INSERT INTO JOTFORM_DATA VALUES ( :content_type,
                                          incoming_payload );
    
    END;

    Table DDL

    You’ll notice a table in the handler code called JOTFORM_DATA. Well, I ended up creating a table of two columns, as seen here:

    1. Content_type
    2. Payload
    CREATE TABLE JOTFORM_DATA 
        ( 
         CONTENT_TYPE VARCHAR2 (4000) , 
         PAYLOAD      VARCHAR2 (4000) 
        ) 
    ;
    Why VARCHAR2(4000)?  Two reasons. First, its just that it seems like VARCHAR2 is catch-all/the most flexible data type for testing like this. Secondly, since the PAYLOAD might be JSON or possibly malformed JSON, if I decide, I can always use an "IS JSON" check constraint. But that can come later.2

    Moving through the workflow

    With the Resource Handler code in place and the table defined, we can test this Webhook and see what it sends across HTTP.

    Next, a look at what my Autonomous database received:

    Okay! So now we have something to work with. And we know at least two things:

    1. The POST request is a multipart/form-data Content-Type, and3
    2. The payload we received appears to be JSON

    Next, we can copy the contents from the PAYLOAD column. And throw it into any number of online JSON Formatting tools, to see what we are working with:

    Final thoughts

    Well, I can already see why somebody might struggle with API integrations (in general, not just here). There is a lot to learn and A LOT of unknowns. Take this actual use case, for instance. Of course, you’d assume this payload to be multipart/form-data. You’d also expect the payload contents to be the JotForm form’s data. However, in this case, the form is pre-processed before it even reaches ORDS or the Autonomous database. So it is a good thing we did some investigation because there is way more “stuff” than we originally anticipated!

    Next, we’ll have to parse through the payload within the payload. For that, we can also use the ORDS Implicit Bind parameters, Automatic Binding, and JSON_VALUE functions. But we’ll save that for the next post. Stay tuned!

    Complete code

    Click for code

    You can find the most up-to-date code here. The code used in this post:

    -- Table DDL
    
    CREATE TABLE JOTFORM_DATA 
        ( 
         CONTENT_TYPE VARCHAR2 (4000) , 
         PAYLOAD      VARCHAR2 (4000) 
        ) 
    ;
    
    -- ORDS Resource Module PL/SQL
    
    BEGIN
      ORDS.ENABLE_SCHEMA(
          p_enabled             => TRUE,
          p_schema              => '[Your username/schema]',
          p_url_mapping_type    => 'BASE_PATH',
          p_url_mapping_pattern => '[same as p_schema in all lowercase]',
          p_auto_rest_auth      => FALSE);
        
      ORDS.DEFINE_MODULE(
          p_module_name    => 'ords_and_jotform',
          p_base_path      => '/oj/',
          p_items_per_page => 25,
          p_status         => 'PUBLISHED',
          p_comments       => NULL);
    
      ORDS.DEFINE_TEMPLATE(
          p_module_name    => 'ords_and_jotform',
          p_pattern        => 'jotform_data',
          p_priority       => 0,
          p_etag_type      => 'HASH',
          p_etag_query     => NULL,
          p_comments       => NULL);
    
      ORDS.DEFINE_HANDLER(
          p_module_name    => 'ords_and_jotform',
          p_pattern        => 'jotform_data',
          p_method         => 'POST',
          p_source_type    => 'plsql/block',
          p_items_per_page => 25,
          p_mimes_allowed  => NULL,
          p_comments       => NULL,
          p_source         => 
    'DECLARE
        incoming_payload CLOB;
    BEGIN
        incoming_payload := :body_json;
        INSERT INTO JOTFORM_DATA VALUES ( :content_type,
                                          incoming_payload );
    
    END;');
    
            
    COMMIT;
    
    END;

    Follow

    And don’t forget to follow, like, subscribe, share, taunt, troll, or stalk me!

    1. What is a Radio button? ↩︎
    2. Directly from the JSON Developer’s Guide Release 23, “You can store JSON data in Oracle Database using columns whose data types are JSON, VARCHAR2, CLOB, or BLOB.” Although, the guide then continues to state that, “Oracle recommends that you use JSON data type, which stores JSON data in a native binary format.” This is an easy change to make early on though. Should I choose to, I can adjust future table columns to accommodate the JSON data type. ↩︎
    3. About the multipart/form-data Content-Type. This section discusses the multipart/form-data Content-Type, but also ExampleBoundaryString why it is used/included. You’ll see that in some of the screenshots throughout. ↩︎

  • ORDS 24.3 Release Highlights

    ORDS 24.3 Release Highlights

    Overview

    There’s plenty to talk about in this release. However, I’m most excited about the performance improvements, ORDS sample applications, and documentation changes. Read on to get the whole story.

    Enhancements

    API Performance

    REST API responses from either AutoREST or customer based modules could see as much as a 30% improvement in response times.

    About a year ago, we introduced (we owe all our progress to Orla though) an internal program to track performance changes/improvements across ORDS APIs quantitatively. I can’t go into too much detail, but here is what I can divulge:

    • Although we use K6 for our performance testing, we are not promoting its use over any other available performance testing solution. There are other great tools available (e.g., Artillery, JMeter, and, of course, k6).
    • Testing is performed nightly against a 23ai database (installed in a PDB); we also include APEX in these tests.
    • For the tests, 250 schemas are created and then populated with various database objects (e.g., Functions, Materialized Views, PLSQL Packages, Sequences, Tables, Triggers, JSON Relational Duality Views, etc.)
      • These schemas are then absolutely hammered with Virtual Users. Users perform actions such as auto-REST enabling objects, creating custom Resource Modules, creating JSON Relational Duality Views, interrogating ORDS Metadata, and performing bulk inserts (BATCHLOAD) GETs, POSTs, etc.
    • These metrics are what we use to track the ORDS quantitative metrics longitudinally.

    So, that’s what we mean by “performance improvements.” Pretty cool, eh?

    ORDS Sample applications

    We have not one but TWO sample ORDS applications for you 😍!

    Flask/Python

    The first is a fully contained LiveLabs sandbox workshop, which can be found here. But if you want to remix the code, check out my repo here (everything is heavily commented; hopefully, this will ease your pain).

    Node.js/React

    Secondly, our development team has created a brand new advanced application. Details are here.

    OAuth2.0 changes

    A consolidation and streamlining of the OAUTH and OAUTH_ADMIN PLSQL packages. The details:

    • We’ve consolidated those mentioned above into these two new packages: 
      • ORDS_SECURITY
      • ORDS_SECURITY_ADMIN
    • The OAUTH and OAUTH_ADMIN PL/SQL Packages have been deprecated by royal decree. However, they’ll still be included until ORDS version 25.3 (this time next year).
    • Creating a client and receiving your Client ID and Client Secret is now streamlined, and Client Secrets can now be rotated (by supporting two active Client Secrets while in rotation).

    Locating the new PL/SQL Packages:

    Finding PL/SQL Packages in Database Actions.

    23ai Boolean

    ORDS now returns BOOLEAN types as JSON TRUE|FALSE properties instead of 0|1.

    What this looks like in practice with various configurations1:

    Oracle Database 23ai + ORDS 24.3
    Oracle Database 23ai + ORDS 24.2
    Oracle DB 21c Enterprise Edition + ORDS 24.3

    1Thank you internet stranger for providing us with this juicy bit of code.

    Mong[ooohhh, no, you didn’t?!] DB API

    • Support for even more Database Administration commands:
      • listIndexes
      • dropIndexes, and
      • optional parameter expireAfterSeconds (which applies to the createIndexes command)
    • The following MongoDB Aggregation Stages are now supported:
    • Users may now specify a JSON Schema validator when creating a collection
    • You may now create MongoDB API connections without TLS connections (Oracle Database 19c and later) can now be made.
    • Users can now set the createIndexes’ “online” option to TRUE to allow DML operations at index creation time.

    In the meantime, here are some resources I found:

    UPDATE: MongoDB API update article (October 10, 2024)

    A brand new article about the latest MongoDB API updates just dropped! Thanks to Hermann for publishing and sharing the latest. Details are here

    Documentation

    Introduced the following new sections:

    • 6.2.4 Using OCI Monitoring Service with Oracle REST Data Services
      • This new section details the configuration of the recently added ords-metrics utility. You can find details on how to set up this monitoring service (to communicate with OCI) here.
    • 3.2 Deploying ORDS with Central Configuration Server
    • Appendix D ORDS Central Configuration OpenAPI
      • Along with the updated docs, we’ve included the OpenAPI spec for creating the endpoints required for a central configuration server (and a special video clip of me retrieving the PL/SQL definitions and the OpenAPI spec in Database Actions).
      • OpenAPI spec doc here.

    Java notes

    In our Release Notes, we claim support for the following JDKs:

    • Oracle Java 11, 17, or 21
    • Oracle GraalVM Enterprise Edition for Java 11
    • Oracle GraalVM Enterprise Edition for Java 17
    • Oracle GraalVM Enterprise Edition for Java 21

    However, this may be confusing regarding Oracle GraalVM Enterprise Editions. You should know that there are currently TWO Oracle GraalVM Enterprise Edition JDKs:

    • Oracle GraalVM Enterprise Edition 20
    • Oracle GraalVM Enterprise Edition 21

    Instead of how we’ve presented, here is another, cleaner presentation of these JDKs:

    Oracle GraalVM Enterprise Edition 20Oracle GraalVM Enterprise Edition 21
    Linux (x86-64): Java 8, 11Linux (x86-64 and aarch64): Java 8, 11, 17
    macOS (x86-64): Java 8, 11macOS (x86-64 only): 8, 11, 17
    Windows (x86-64): Java 8, 11Windows (x86-64 only): 8, 11, 17
    Oracle GraalVM Enterprise Edition details

    So when you are choosing your JDK (to use with ORDS), make sure you consider your platform and use cases. Details on using GraalVM with ORDS here.

    fin

    This concludes the release notes supplement.

    This space ⬇️ left intentionally blank.

    Follow

    And don’t forget to follow, like, subscribe, share, taunt, troll, or stalk me!

  • Extending ORDS with a Java plugin

    Extending ORDS with a Java plugin

    About this article

    This article is a walkthrough and overview of setting up the example Java plugin that ships with ORDS. These plugins allow you to add functionalities and capabilities above what is possible with a standard ORDS configuration.

    In this example, the plugin allows you to use a name as a query parameter in an HTTP request. What you receive in response is a greeting from the schema plus the name you used in that HTTP request. Like this:

    I know what you are thinking…so what? And for about a day and a half, I felt the same. Eventually, it dawned on me that since this plugin relies on APIs from the Oracle REST Data Services Java API Reference, you can create a plugin for anything in that book. I’ll have to spend time in another article, stepping through the Java code. I’m realizing that you don’t need to know Java fluently to understand what is happening with this application. So, if you are a Java newbie (like me), this is a great way to get acquainted with the language.

    Chapter 11 Extending ORDS Functionality with Plugins

    Below, you’ll see the exact text from Chapter 11 Extending ORDS Functionality with Plugins of the ORDS Developer’s Guide. If you want to follow along, you won’t have to switch between the official docs and this blog post.

    Further still, I’ve broken up the steps in Section 11.1 Plugin Demonstration Example into individual sections. I’ve added some commentary and images throughout. Some directions were challenging for me, and pictures are always helpful.

    Oh! And there’s a special “How the hell do I set up Ant?!” section, too. Since it took me about 30 mins to figure out how to download, install, and configure the damn thing. It would be best if you didn’t have to suffer like I did 😘.

    The unadulterated ORDS docs

    HINT: Click the expand/collapse arrow to view the full text.
    11 Extending ORDS Functionality with Plugins

    This chapter explains and provides examples on using ORDS plugin framework.

    ORDS has a plugin framework that allows you to add your own custom functionality into the ORDS web application. Plugins can be added to the ORDS runtime by placing the jar files in the lib/ext directory. The ORDS distribution contains the source for example plugins. The plugin examples can be built using Apache ant, a software tool used for automating the build processes.

    11.1 Plugin Demonstration Example

    This section shows how you can locate and build a plugin demonstration example.

    The plugin-demonstraion example is at examples/plugins/plugin-demo location and contains the source for a HttpServlet that gets a database connection injected at runtime. The servlet uses that JDBC database connection to run a query in the database and return a response at runtime.

    Perform the following steps to build and use the demonstration example:

    1. Change the directory to examples/plugins/plugin-demo
    2. Run ant to build the examples/plugins/plugin-demo/built/plugin-demo.jar file
    3. Copy the plugin-demo.jar to the ORDS distribution lib/ext directory and start an ORDS instance.
    4. Invoke the servlet using the following URL pattern:http://server/ords/schema/demos/plugin?who=somebody
      1. For example: http://localhost:8080/ords/hr/demos/plugin?who=scott where ORDS is configured with a default pool and HR is an alias for a REST Enabled Schema in that database.

    The details of developing and deploying Java based plugins is available in the Oracle REST Data Services Java API Reference book.

    See Also:

    Route Patterns Specification

    Getting Started Guide

    Developer Guide

    Chapter 11.1 Steps, expanded

    Locate the directory

    Change the directory to examples/plugins/plugin-demo.

    This step assumes you have Ant installed and are ready to build the application. In other words:

    1. You’re in your Terminal, Shell, or Command Prompt.
    2. You have issued the cd command, so you are in the /plugin-demo directory, and
    3. You’re about to run the ant command on the command line.

    HOWEVER…If you don’t have Ant installed, this exercise won’t work. Let’s get Ant.

    Get Ant

    Introducing a special section dedicated to making Ant acquisition much easier!

    You’ll need to visit the Apache Ant main page. But don’t waste your time navigating to the Downloads page. It makes no sense.

    Just click Manual (located under the Documentation section). A new page will appear, and you’ll see a Table of Contents. Click “Installing Apache Ant.” Then, click “Getting Ant.”

    Once you are at the “Getting Apache Ant” page, you’ll see five steps. For Step 1, I’ll assume you have Java installed.

    Also, if own a computer, one that you bought and didn't build, there's like a 99% chance you have Java installed. It just might not be the latest version.

    In Step 2, we need the Binary Distribution. Of the four links there, I chose the .zip file. It seemed the most normal (I haven’t seen .tar files since working on IBM Z 😵‍💫).

    Step 3 states to unzip and then place the directory somewhere. I’ll show you what I did in a second.

    Step 4 instructs you to do some things and references the Setup page for details.

    If you don’t understand any of the instructions in Step 4, please feel free to click the Setup link. You’ll be presented with even more steps that don’t make sense.

    Let me show you how I have this all set up. If you are on a Mac, you can see how I have this bin directory and ANT_HOMEvariable set up. I open a new Terminal and issue the following command: open .zprofile

    At the bottom of the file, you’ll see how I have Apache Ant set up for both bin and ANT_HOME. Set yours up like this: save the file, then restart your Terminal.

    Wut did I just do? My idiot's take...After doing this, you can now execute the ant command in any terminal. And you've also "pointed" to all the required dependencies (i.e., ANT_HOME) for Ant to make the magic happen.

    After restarting, you can issue the ENV or env command to review your environment settings. Here, you’ll see the Ant bin is in my PATH. You can now build with Ant 👍🏻.

    And to answer the question, “Where do I put the Ant files?” I placed them in my Library on my Mac.

    The Apache Ant docs mention two commands for testing to see if Ant is working as expected. The first command ant can be issued anywhere.

    However, I assume if you issue this command in a directory where source files are waiting to be built, then you'll actually end up building a new Java app. I believe this is why they recommend issuing this command in a new shell. It assumes you don't have files there, waiting to be built.

    Secondly, you can issue the ant -version command. If you have Ant installed, you’ll see the version information.

    Okay! You now have Ant; let’s get back to the colony 🐜🐜🐜.

    Build with Ant

    Run ant to build the examples/plugins/plugin-demo/built/plugin-demo.jar file

    README: The plugin-demo directory is located in your ORDS product folder. This step assumes you have ORDS installed and configured for Standalone, Weblogic, or an Apache Tomcat server.

    Next, “cd” to the plugin-demo directory. I am lazy, so I right-click on the folder and choose to open a “New Terminal at Folder.”

    The official instructions state that the source files for this Java plugin are contained in this folder. You do not need to modify these files for this to work. They are shipped with every ORDS version, so they will always be there, even when you upgrade (have you upgraded to the latest ORDS?).

    You are not losing your mind. The build literally takes about 5 seconds. You’ll see a printout like the image below. You only need to be concerned with the dist: row at this stage. That is your plugin, and it is ready to be deployed.

    Jar configuration

    Copy the plugin-demo.jar to the ORDS distribution lib/ext directory

    Navigate back to the plugin-demo directory. While you’re there, you can expand the sub-directories. You will see the original src directory and the new built directory, with its sub-directories and files. Copy the plugin-demo.jarfile.

    Return to your ORDS product folder and place the plugin-demo.jar file in the lib/ext directory. Just so we are clear, there is another lib directory nested in examples/plugins/lib. You DO NOT want to move the jar to that folder; you must go to the ords product folder/lib/ext directory.

    Starting ORDS

    Start an ORDS instance with the ords serve command.

    Okay, we are almost there. I have a couple of ORDS configurations here. One is for working in ORDS Standalone mode with a 23ai database via a Podman container (the Podman image can be found here), and another is a Customer-managed ORDS installation (identified by the moviestream23ai_medium database pool name) for working with my Autonomous database (in OCI).

    ORDS customer-managed installation instructions can be found here. 

    Test the plugin

    Invoke the servlet using the following URL pattern: http://server/ords/schema/demos/plugin?who=somebody.

    For example: http://localhost:8080/ords/hr/demos/plugin?who=scott where ORDS is configured with a default pool and HR is an alias for a REST Enabled Schema in that database.

    Now that ORDS is up and running, we can do some testing. In this first example, I challenged my creativity by using the name “Chris” for the test. Here, I’m issuing a request to the plugin. Of which it successfully returns the following message:

    [ORDS REST-enabled schema] says hello to: [the name passed as a query parameter in the original request].
    
    OR
    
    ORDSTEST says hello to: Chris

    Testing with the Podman container:

    And with the Customer-managed ORDS with Autonomous database installation:

    Success! I am me!

    Next steps

    You’re probably asking yourself, “Now what?” Well, here is what I would do:

    1. Review the above steps again and on the docs for building this Java plugin.
    2. Look at our support for JavaScript plugins and the extra configuration step if you use GraalVM for your JVM.
    3. Start learning Java. The two resources I’ve been reading in my spare time:
    4. Check out Peter’s ORDS plugin example too
    5. Make your own plugin and share it with me so I can try it out 😬

    Oracle CloudWorld 2024 updates

    And finally, yes, we are fast approaching this year’s CloudWorld conference. If you didn’t see my last post on our teams’ activities, review it (and hopefully register for at least some of our sessions). Details are here. And that’s all for now!

    Follow

    And don’t forget to follow, like, subscribe, share, taunt, troll, or stalk me!

  • ORDS 24.2 Release Highlights

    ORDS 24.2 Release Highlights

    NOTE: This can be thought of as a "companion piece" to the official ORDS release notes 🤓.

    What is new in Oracle REST Data Services 24.2?


    New HTTPS Response Status Codes

    In response to various support requests and internal feedback, we've expanded on the existing ORDS Status Codes (the current list can be found here)!
    Why? The TL;DR is that in many cases, the error or exception caught was being "bucketed" into a category of codes, like a 400 Bad Request or a 500 Internal Service Error. While true, we can be more explicit with some of these error codes. Now, we just have to ask, at what level do we stop?!

    Ahem, this sounds like our entire dev team is lazy; it’s quite the opposite. Let me illustrate what’s actually going on with an example from one of our related tickets.

    Say a user receives a 503 Service Unavailable server error response. In their Java logs (trace files, access logs, etc.), they also observe an oracle.net.ns.NetException error. While classifying this as a 503 is accurate, we can do better. For instance, you may encounter an oracle.net.ns.NetException with any of the following ORA codes:

    ORA-12514 TNS: The listener does not currently know of the service requested in the connect descriptor
    ORA-12506 TNS: listener rejected connection based on service ACL filtering
    ORA-12529 TNS: connect request rejected based on current filtering rules
    ORA-12516, TNS: The listener could not find an available handler with a matching protocol stack

    In this example, moving forward, instead of receiving a 503, you’ll now receive a 571 server error response.

    We’ve done this for several other scenarios, too! Why keep everything generic, as a 503, when we can be more discrete and specific with the information provided? It makes no sense. So, by shifting some of these exceptions to unique response codes, we make it easier for you to identify what is happening.

    What if everything was either 200, 400, or 500? Could you imagine?! It would be nearly impossible to quickly identify what is happening in your stack.

    ORDS Central Configuration

    ORDS now supports deployment in a Central Configuration type deployment.

    This one is big. The short version is that we’ve made it so you can dynamically start up and shut down individual ORDS “nodes.” This requires three main pieces, but I’ll try to be brief.

    This all assumes you’ve already configured ORDS globally (i.e., you have a global.xml file) as well as your database pools (i.e., the default.xml, database_pool_one.xml, database_pool_two.xml, etc. files). But the idea is such that you’ll have stored in a central Vault/Secrets storage/Key Store your:

    1. ORDS global configuration – which will be in a JSON format (basically a JSON version of the global.xml file)
    2. ORDS database pool configuration/s – the individual configuration files for your ORDS “nodes” (again, just JSON versions of your database-pool.xml files)
    3. A mechanism for the ORDS_PUBLIC_USER to authenticate with the Vault or Keystore (this could also be something as simple as an ORDS webhook that has been protected with an OAuth2.0 client)

    *And a conditional fourth item, the makestore or orapki utilities (for creating SSO Wallets).

    One way to “do” this (we’ve tested this internally; we just can’t, and won’t endorse, a “one-size fits all” method) is to store your “secrets” in a vault-type service and retrieve them securely and dynamically.

    NOTE: The vault contains "secrets," including your global configuration and database pool files. Each of those secrets might be behind an OAuth 2.0-protected endpoint/s.

    Separately, and before starting up ORDS, you’d create an SSO Wallet with the credentials for the ORDS_PUBLIC_USER, its password, and additional hostname information (all can be found in our documentation). You’d also choose where to store that Wallet.

    Then, before launching ORDS, you’d include two Java options:

    1. Your wallet location (so when ORDS starts up, it is aware of the location of the Wallet and the credentials) and
    2. The REST endpoint (the URI) for your global configuration

    Authentication can take several forms here. You can use Basic Authentication (but…don’t, even though we support it, perhaps for testing, but please don’t use it for production), JWTs, or OAuth 2.0. Once the ORDS_PUBLIC_USER has authenticated with its credentials/token, ORDS can acquire its global configuration.

    When that HTTP request comes across ORDS, the database pool name can be passed as a header value (we can also read the database pool name if it is appended to the beginning of the URL) and used as a “search” value to retrieve the relevant database_pool-config.json file.

    That is ORDS Central Configuration in a nutshell. And since I lost you, I’ll have to write a follow-up blog on this. I don’t blame you; conceptually, it’s tough to envision, but it’s pretty simple in practice. The basic components are laid out in our documents.

    OCI Monitoring of ORDS

    ORDS now provides a create_alarms.sh script to create ORDS alarms using OCI Monitoring Services, which can be found in the [ords product folder]/examples/ords-metrics directory.

    How shall I explain this without getting in trouble with our legal department…? This is being rolled out globally right now for the Oracle Autonomous database. Before this ORDS release, errors or exceptions would be caught and streamed to an OCI metrics dashboard. And you might have seen an ORDS-related exception with a 404 or a 503. Unfortunately, the root-cause analysis would have already been off to a bad start. Because most of these exceptions aren’t just 404s or 503s. There had yet to be a mechanism to make some of these exceptions more discrete. And that is where this create_alarms.sh script enters the chat.

    If you review the now-included create_alarms.sh file, you can get an idea of what will now be streamed to your OCI Metrics/Monitoring Explorer. We’ve made it so the ORDS access logs and more appropriate response codes can be streamed to OCI. This makes root-cause analysis and troubleshooting far more straightforward. So if you elect to configure a customer-managed ORDS node, then be sure to take advantage of this new capability.

    A new standalone configuration option

    New standalone.access.log.retainDays configuration option for ORDS standalone. 

    Users can now customize the number of days before access log files are overwritten. The default is 90 days, but you can now select the exact number. This, along with numerous other standalone settings can be found in the Understanding Configurable Settings appendix of our ORDS Installation and Configuration Guide.

    Java options in the ORDS CLI

    Users may now include Java Options parameters when executing ORDS CLI commands. 

    For instance, a user may execute the following command:

    ords --java-options "-Djava.util.logging.config.file=logging.properties"

    Or, maybe you are testing the new ORDS Central Configuration strategy. In that case, you might want to include something like this when starting ORDS:

    ords --java-options "-Dconfig.url=//localhost.8080" serve

    In the above example, you’ve told ORDS to start up, and then, first thing, go to the target URL to retrieve ORDS’ global configuration settings.

    You may wonder about JAVA_OPTIONS and JDK_JAVA_OPTIONS; how are those settings impacted? Well, here are some essential details:

    1. ords --java-options only apply to the current ORDS execution.
    2. In order of precedence, options will be picked up like this:

    JAVA_OPTIONS are of the highest precedence >>> then >>> ords --java-options >>> then >>> JDK_JAVA_OPTIONS

    INFO: Where conflicts arise, ORDS will pick up and use the left-most option.

    Jetty 10.0.21

    ORDS standalone updates the embedded Jetty Web Sever version 10.0.21.

    There’s not much to say here, but if you’d like to learn more about Jetty, I’m including the Operations and Programming guides. When using ORDS in standalone mode, the idea is that Jetty is just there—and it just works.

    We’ve architected Standalone mode so that any ORDS configuration can be achieved via the ORDS CLI or by directly manipulating the XML configuration files (while you can do this, you shouldn’t; you might mess up one of the properties in the files). Designing ORDS Standalone mode this way makes it so you don’t really need to do anything Jetty-related. I’m just including the docs if you want something to read on your lunch break 🤪.

    Database Actions’ Data Pump

    Data Pump allows you to delete jobs and their files from within the Database Actions Data Pump UI. And the Data Pump Import Wizard features some major upgrades and visual enhancements. 

    Users who have DBMS credentials to access Oracle Cloud Services (via the DBMS_CLOUD.CREATE_CREDENTIAL PL/SQL procedure) can now perform Data Pump Imports from OCI Buckets (previously, it was Resource Principals only). And we’ve added the following abilities/changes:

    1. Buckets from any Compartment level are now accessible
    2. Auto-generated Import Patterns for DMP Files are now included
    3. Automatic mapping for Schemas and Tablespaces is now present
    4. A new “Append Timestamp to Log and Job Names” toggle option has been added

    NDJSON files

    Users can now import Newline Delimited JSON (NDJSON) or .ndjson files into the SQL Worksheet.

    This actually arose from a bug we encountered. After some investigation, the user attempted to import a “Newline Delimited” JSON document. I’m not familiar with this document type, but cursory research reveals the following:

    There is currently no standard for transporting instances of JSON text within a stream protocol apart from [Websockets], which is unnecessarily complex for non-browser applications.

    A common use case for NDJSON is delivering multiple instances of JSON text through streaming protocols like TCP or UNIX Pipes. It can also store semi-structured data.

    NDJSON spec Github

    If you’d like to learn more about this specification, I recommend visiting this site. It doesn’t seem affiliated with the project, but it is very informative compared to the official NDJSON GitHub page.

    Are you using NDJSON now? Or did you just learn of its existence? Do you think you’d start using it, or do you have any use for it? Let me know. I’m curious about the potential applications.

    New Walkthroughs (Tours) for Charts and Data Modeler

    Updates to the Database Actions Launchpad. 

    We continue to refine the Launchpad, and these are but two more examples. The Data Modeler and Charts pages have Walkthrough Tours (and documentation too):

    What are your thoughts on the new Launchpad? Do you use the “pin” feature? Is it intuitive? What else could we include or improve? Or is it perfect (it’s perfect, isn’t it? …I knew it!)

    Java

    ORDS requires Oracle Java or Oracle GraalVM Enterprise Edition 11, 17, or 21 to run ORDS 24.2.

    If you intend to do anything GraphQL-related, use GraalVM 17. Instructions for downloading can be found here. Just make sure you set your JAVA_HOME to GraalVM 17 so that when ORDS starts up, it does so with GraalVM 17! At this time, ORDS GraphQL support only works with GraalVM 17.

    That’s it for this release. Download the latest and enjoy!

    Oh, and if we missed your enhancement this time, let me know. We’ll add it to one of our sprints!

    Follow

    And don’t forget to follow, like, subscribe, share, taunt, troll, or stalk me!

  • Python script to retrieve objects from Oracle Cloud Bucket

    Python script to retrieve objects from Oracle Cloud Bucket

    For…reasons, I needed a way to retrieve all the .CSV files in a regional bucket in Oracle Cloud Object Storage, located at this address:

    https://objectstorage.us-phoenix-1.oraclecloud.com/n/dwcsprod/b/moviestream_data_load_workshop_20210709/o

    You can visit it; we use it for one of our LiveLabs (this one), so I’m sure it will stay live for a while 😘. Once there, you’ll see all the available files in that bucket:

    FYI: A bucket is one of the three core components of Oracle Cloud's Object Storage. The other two are Objects and Namespaces (more details at Object Storage FAQs).

    In this case, there were more than just .CSV files; the script I created will also download those. But, of course, your specific situation may vary. Onto the Python script!

    The code

    Allow me to first address a few considerations and limitations:

    1. There is probably a way to create a function that accepts a parameter to make this even more elegant and compact. I’m not “there” yet. So, if you have a better solution, please comment, and I’ll amend my code!
    2. I still haven’t learned error and exception handling in Python. So anytime you see me “code in” print(...) that is basically me attempting to check to see if what I think will happen actually happens.
    3. I’m not sure if my range() and len() practices are appropriate, but it works, so I’m going with it!
    4. Don’t ask me how I even found out about the webbrowser library. I must have found it on a forum, or StackOverflow (hate that I can’t attribute credit to my savior).

    The code for real

    import requests
    import json
    import webbrowser
    
    url = 'https://objectstorage.us-phoenix-1.oraclecloud.com/n/dwcsprod/b/moviestream_data_load_workshop_20210709/o/'
    
    r = requests.get(url)
    
    data = r.json()
    # print(data)
    
    newlist = []
    for name in data['objects']:
        newlist.append((name['name']))
    
    # print(newlist)
    length = len(newlist)
    newurl = []
    
    for i in range(length):
        newurl = url + newlist[i]
        webbrowser.open(newurl, new=0, autoraise=True)

    As you can see, with spacing, it’s less than 25 lines of code. I’m using a total of three libraries: requests, json, and webbrowser. The first two libraries you are probably familiar with, the third maybe not so much. Webbrowser() is great, when used correctly, the code will automatically open a new browser tab and execute whatever it is told to do (in this case, go to a URL and issue a GET request). Make sense?

    Not to worry, I’ll break this down into smaller blocks, to better understand what is happening.

    Lines 1-10

    import requests
    import json
    import webbrowser
    
    url = 'https://objectstorage.us-phoenix-1.oraclecloud.com/n/dwcsprod/b/moviestream_data_load_workshop_20210709/o/'
    
    r = requests.get(url)
    data = r.json()
    # print(data)

    I’ve imported the three relevant libraries (remember if you don’t have these libraries, you can use pip and perform a pip install to grab them). And I assign URL equal to the target bucket. And from here I perform my first GET request. I do this for two main reasons:

    1. to remind me of the structure of the JSON, and
    2. because I am about to loop through all the available files in this bucket (so I need to capture them somehow)

    Since I’ve essentially assigned the value of r.json() equal to data, I can now print(data) to visually inspect the….um….data. After executing lines 1-10, I can see the corresponding output in my terminal:

    Now that I know this is working (because of my manual check), I can proceed to the next step.

    Lines 12-16

    Executing this code…

    import requests
    import json
    import webbrowser
    
    url = 'https://objectstorage.us-phoenix-1.oraclecloud.com/n/dwcsprod/b/moviestream_data_load_workshop_20210709/o/'
    
    r = requests.get(url)
    
    data = r.json()
    print(data)
    
    newlist = []
    for name in data['objects']:
        newlist.append((name['name']))
    
    print(newlist)

    Will yield a new list:

    But how you ask? Good question, lots of trial and error. In short, I created an empty list, named newList. And later populated it with “stuff”. In normal people speak, lines 13-14 say the following,

    “There is an item, actually many items in the JSON payload we received from our GET request. For simplicity, lets call each of those items name, since ultimately what we want are file names. And we’ll use bracket notation to ‘drill’ down into more specific sections of the original JSON payload. And since we want just the file name (values), we want all the stuff that is in objects (if that doesn’t make sense review the lines 1-10 section again). We are then going to add to newList only the names (aka values) of the files. And these lines of code are what help us to populate the newList list.

    Maybe by this stage, you’ve caught on to what I’m trying to do. If not, I won’t spoil it for you 🙃.

    Lines 18-19

    And if you don’t quite “get it” yet, not to worry, it’ll click here in a second. Next, I need to see how many items are in this newlist I just created. Here is the code that is responsible for making that happen:

    import requests
    import json
    import webbrowser
    
    url = 'https://objectstorage.us-phoenix-1.oraclecloud.com/n/dwcsprod/b/moviestream_data_load_workshop_20210709/o/'
    
    r = requests.get(url)
    
    data = r.json()
    # print(data)
    
    newlist = []
    for name in data['objects']:
        newlist.append((name['name']))
    
    # print(newlist)
    
    length = len(newlist)
    print(length)

    Here is the result in my terminal:

    It looks like I have 85 elements, things, stuffs in my list. I’ll use that knowledge for the final steps in my script.

    Lines 21-26

    import requests
    import json
    import webbrowser
    
    url = 'https://objectstorage.us-phoenix-1.oraclecloud.com/n/dwcsprod/b/moviestream_data_load_workshop_20210709/o/'
    
    r = requests.get(url)
    
    data = r.json()
    # print(data)
    
    newlist = []
    for name in data['objects']:
        newlist.append((name['name']))
    
    # print(newlist)
    
    length = len(newlist)
    # print(length)
    
    newurl = []
    
    for i in range(length):
        newurl = url + newlist[i]
        webbrowser.open(newurl, new=0, autoraise=True)
        print(newurl)

    Line 21, you’ve seen this before (like 2 mins ago).

    However, lines 23-25 are new. If you recall my “considerations and limitations” section. I bet there is a better way to do this (or all of this actually), but this worked for me, so I’ll keep it. In short, we know we have 85 “iterables” (that’s the for i seen in line 23) and the range(length) portion simply says, “Begin at 0 and then keep going till you reach the entire length of…length()…which in this case is 85).”

    Next, I take the original URL (from line 5 in the code) and add each element of the newList to the end, making a newurl. From there, we open a web browser (new tabs actually) and visit that new amended URL aka newurl (I think more appropriately this is a URI, no?).

    And for the visual, a video of the entire process (no audio):

    And finally, for my own gratification, I’ll print out the newurl list just so I can marvel at my work. The new URLs:

    Now that I have these individual files, I can simply drag and drop them into my tables (in Database Actions obviously).

    As always, this and other code can be found in my blog repo. I hope this comes in handy someday! That’s all for now 😀!

    Follow

    And don’t forget to follow, like, subscribe, share, taunt, troll, or stalk me!

  • Build an ORDS API Resource Module, GET request with JavaScript fetch, display in HTML

    Recap

    This post is a continuation of a previous one, which can be found here. In this post, I’ll:

    If you are coming from the previous related post, then you’ll recall I used the following SQL query:

    My next step is to take this SQL and bring it to the REST Workshop, where I’ll turn it into an API.

    REST Workshop

    There are several ways you can navigate to the REST Workshop. Typically, I return to the Database Actions LaunchPad. From there, I select REST.

    The Handler code

    I've already created my Resource Module, Template, and Handler. I kept everything default, with no authentication enabled.

    The only thing I changed was the SQL query. I removed the final line, fetching the first 10 only. I want to be able to control the pagination of the API. If I were to keep that last line, this eventual endpoint would always only return the first 10 rows. And what if I want the next ten rows thereafter? Well, if I hard-code this, then I can’t really make that work. So, I chose to leave it open-ended.

    Technically, it is NOT open-ended because I retained the default pagination of 25. But, by removing that fetch first 10 rows condition, I can now fetch ALL rows that fit those parameters (in increments of 25).

    If I visit this new endpoint, it will appear like this:

    And if I collapse the items, you’ll see something that is EXTREMELY confusing. If I removed that fetch first 10 rows condition in the original SQL query, then why do we see a limit and offset of 10?

    The answer is because I actually set the Items Per Page equal to 10 (in the Resource Handler). This is the REST equivalent of a dirty joke. Consider yourself roasted…

    JavaScript

    With that endpoint live, I can take the API and drop it into some sample JavaScript and HTML code.

    JavaScript and HTML

    I learned a great deal about this JavaScript by reviewing this YouTube video. That is where I learned how to map through the items of my ORDS payload. And there was a refresher on JavaScript string interpolation (with template literals) too!

    PAUSE: Don't be too intimidated by string interpolation and template literals! Read the link I included, and take your time. If you are coming from Python, its similar to Jinja (when using Flask) and f-string literals 🙃.

    You can see that I’m using the map() constructor to iterate through all the data in my JSON payload. Remember, this was the payload in the items portion of my endpoint!

    I believe the item in list.map((item) is a reference to an individual item inline 4’s data.items. The reason why I think this is because if I change the items in lines 7-10 in my JavaScript to something random, like the name bobby, things start to break:

    However, if I change everything back to item, and start the live server in VS Code, I’ll be met with the following rendering:

    That’s it, though. Combining the ORDS API, the Fetch API, simple JavaScript, and HTML will allow you to create this straightforward web page.

    Reviewing Inspector, Console, Network

    I also have a few more screenshots, one each for the HTML Inspector, Console Log, and Client/Server Network. All of these show what is happening under the covers but in different contexts.

    Inspector

    In the Inspector, you can see how the JavaScript map() constructor plus the document.querySelector() in line 18 of the JavaScript code work in tandem with line 12 of the HTML script to display contents on the page:

    Console

    Here, you can see the items in the Console. This is because we added console.log(item)in line 19 of the JavaScript code.

    Network

    Finally, you can see the 200 GET request from our ORDS API. Then, on the far right of the screen, you can see the JSON payload coming from that same ORDS endpoint.

    Admittedly, the way the “Cast” is displayed is not correct. That is yet another array of cast members. And I’ve yet to learn how to structure that correctly. So, if you are reading this, and you know, let me know!

    Finally, all the code you’ve seen in this post can be found in my moviestreamjs github repository.

    That’s all for now!

    Follow

    And don’t forget to follow, like, subscribe, share, taunt, troll, or stalk me!

  • Create a view from a JSON Collection and REST-enable it with ORDS

    The DBMS_CLOUD PL/SQL Package

    You can use this PL/SQL procedure (in the DBMS_CLOUD package) along with the file_uri_list URL (seen in the code below) to create and then add JSON documents to a JSON Collection (good info on JSON Collections in the Oracle database).

    In this example, we call this collection Movie_Collection.

    -- create and load movie json collection from a public bucket on object storage

    begin
    dbms_cloud.copy_collection (
    collection_name => 'MOVIE_COLLECTION',file_uri_list => 'https://objectstorage.us-ashburn-1.oraclecloud.com/n/c4u04/b/moviestream_gold/o/movie/movies.json',
    format => '{ignoreblanklines:true}');
    end;
    /

    Create a view

    With that JSON collection in place, I can then create (aka I’ll continue stealing this code from the same LiveLab) a View of it using the following SQL code:

    /* Create a view over the collection to make queries easy */

    create or replace view movie as
    select
    json_value(json_document, '$.movie_id' returning number) as movie_id,
    json_value(json_document, '$.title') as title,
    json_value(json_document, '$.budget' returning number) as budget,
    json_value(json_document, '$.list_price' returning number) as list_price,
    json_value(json_document, '$.gross' returning number) as gross,
    json_query(json_document, '$.genre' returning varchar2(400)) as genre,
    json_value(json_document, '$.sku' returning varchar2(30)) as sku,
    json_value(json_document, '$.year' returning number) as year,
    json_value(json_document, '$.opening_date' returning date) as opening_date,
    json_value(json_document, '$.views' returning number) as views,
    json_query(json_document, '$.cast' returning varchar2(4000)) as cast,
    json_query(json_document, '$.crew' returning varchar2(4000)) as crew,
    json_query(json_document, '$.studio' returning varchar2(4000)) as studio,
    json_value(json_document, '$.main_subject' returning varchar2(400)) as main_subject,
    json_query(json_document, '$.awards' returning varchar2(4000)) as awards,
    json_query(json_document, '$.nominations' returning varchar2(4000)) as nominations,
    json_value(json_document, '$.runtime' returning number) as runtime,
    json_value(json_document, '$.summary' returning varchar2(10000)) as summary
    from movie_collection
    ;

    Here is what the code looks like in the SQL Worksheet (a part of Database Actions).

    With that View created, you could go one step further and query with even more specific SQL. In this case, I’ll query the View but exclude any entries where a movie cast does not exist:

    Select
    title,
    year,
    gross,
    cast
    from movie
    Where cast is not null
    Order By 3 DESC nulls last
    Fetch first 10 rows only;

    Here is the SQL, with the Script Output below:

    ORDSify it®

    With ORDS, we can REST-enable pretty much any database object we want.

    I have objects, Greg. Can you REST-enable me?

    But after spending a few minutes with this collection, I found the MOVIE View to be the easiest, most sensible object to highlight. It’s a straightforward process, with primarily right-mouse clicks.

    From the Navigator Panel, select Views from the list of available database objects.

    Then, right-click on the Movie View, and select REST, then Enable.

    A slider will appear, for this example, I’ll keep everything default and click Enable (no authentication either, I’m being lazy).

    A Confirmation notification will appear in the upper right-hand corner of the browser 👍🏻.

    Navigate back to the Movie View, and select the cURL command option. Twasn’t there before, tis now!

    Select the GET ALL endpoint, and copy the URI. JUST the URI portion!

    Open a new browser tab, or window, and navigate to the URI. You’ll see everything in the Movie View now!

    Why a view?

    Yeah, good question. There is probably a performance improvement with using Views. Based on what I’m finding, I can’t definitively say, but they require zero storage, so that’s at least a savings on some resource somewhere. Additionally, I think the customization is pretty compelling, too. What do I mean by that? Well, allow me to elucidate:

    Views can provide a different representation (such as subsets or supersets) of the data that resides within other tables and views. Views are very powerful because they allow you to tailor the presentation of data to different types of users.

    Oracle Database Administrator’s Guide Release 23

    In this case, the Movie View returns everything found in the collection. I can subset this even further though; by taking that SQL query and REST-enabling it, too. I will, but in a future post.

    For now, I’ve left you with an easy way to REST-enable a View (In this case, based on a JSON Collection) that resides in your Autonomous Database.

    If you want to try the LiveLab (which you should, as it’s easy and VERY informative), go here. You’ll need an Always Free OCI account, too (so you can provision an Autonomous Database). You can sign up here.

    Oh, and we are entering into the season of Cloud World 2024, so mark your calendars 🤪!

    That’s all for now 😘.

    Follow

    And don’t forget to follow, like, subscribe, share, taunt, troll, or stalk me!

  • Oracle Database REST APIs and Apple Automator Folder Actions

    Oracle Database REST APIs and Apple Automator Folder Actions

    The plan was to create an ORACLE REST endpoint and then POST a CSV file to that auto-REST enabled table (you can see how I did that here, in section two of my most recent article). But, instead of doing this manually, I wanted to automate this POST request using Apple’s Automator application…

    Me…two paragraphs from now

    Follow along with the video


    The plan

    I did it. I went so far down the rabbit hole, I almost didn’t make it back alive. I don’t know when this ridiculous idea popped into my head, but it’s been well over a year. Until now, I either hadn’t had the time or the confidence to really tackle it.

    The plan was to create an ORACLE REST endpoint and then POST a CSV file to that auto-REST enabled table (you can see how I did that here, in section two of my most recent article). But, instead of doing this manually, I wanted to automate this POST request using Apple’s Automator application.

    The use case I made up was one where a person would need to periodically feed data into a table. The data doesn’t change, nor does the target table. Here is an example of the table I’m using:

    bank transfer table structure chris hoina, senior product manager, ords, oracle rest, database tools, sqlcl, sql developer, oracle rest
    The basic structure of the Bank Transfers table

    And the DDL, should there be any interest:

    CREATE TABLE ADMIN.BANK_TRANSFERS
    (TXN_ID NUMBER ,
    SRC_ACCT_ID NUMBER ,
    DST_ACCT_ID NUMBER ,
    DESCRIPTION VARCHAR2 (4000) ,
    AMOUNT NUMBER
    )
    TABLESPACE DATA
    LOGGING
    ;

    Once this table was created, I auto-REST enabled the table and retrieved the complete cURL Command for performing a Batch Load request. Remember, we have three examples for cURL Commands now, I chose Bash since I’m on a Mac:

    batchload curl command for bank transfers table, chris hoina, senior product manager, ords, oracle rest, database tools, sqlcl, sql developer, oracle rest
    Retrieving the the Batch Load cURL Command

    Once I grabbed the cURL Command, I would temporarily save it to a clipboard (e.g. VS Code, TextEdit, etc.). I’d then create a new folder on my desktop.

    ords curl post new folder for batch load automation, chris hoina, senior product manager, ords, oracle rest, database tools, sqlcl, sql developer, oracle rest
    The newly created ords_curl_post folder

    How I actually did it

    I’d then search via Spotlight for the Automator application. Once there, I’d choose Folder Action.

    choose automator folder action, chris hoina, senior product manager, ords, oracle rest, database tools, sqlcl, sql developer, oracle rest
    Choosing Folder Action for this automation
    HEY!! README: I'm going to breeze through this. And it may seem like I am well-aquainted with this application. I am not. I spent hours debugging, reading through old StackExchange forums, and Apple documentation so I could share this with you. There is a ton more work to do. But bottom line, this thing works, and its something that is FREE and accessible for a lot of people. You could have a TON of fun with this stuff, so keep reading!

    There’s no easy way to get around this, but to get really good at this, you’ll just need to tinker. Luckily, most of these automation modules are very intuitive. And there is a ton of information online on how to piece them all together.

    Automator 🤖

    All of these modules are drag-and-drop, so it makes it easy to create an execution path for your Folder Action application. Eventually, I ended up with this (don’t worry, I’ll break it down some, a video is in the works for a more detailed overview):

     complete folder action automation, chris hoina, senior product manager, ords, oracle rest, database tools, sqlcl, sql developer, oracle rest
    Complete Folder Action automation for the ORDS Batch Load request

    The modules

    The modules I’m using are:

    • Get Specified Finder Items
    • Get Folder Contents
    • Run Shell Script (for a zsh shell, the default for this MacBook)
    • Set Value of Variable
    • Get Value of Variable
    • Display Notification

    You can see at the very top, that I have to choose a target folder since this is a folder action. I chose the folder I created; ords_curl_post.

    Get Specified Finder Items and Get Folder Contents

    The first two modules are pretty straightforward. You get the specified finder items (from that specific folder). And then get the contents from that folder (whatever CSV file I drop in there). That will act as a trigger for running the shell script (where the filename/s serve as the input for the cURL Command).

    PAUSE: I must confess, I had essentially ZERO experience in shell scripting prior to this, and I got it to work. Its probably not the prettiest, but damn if I'm not stoked that this thing actually does what it is supposed to do.

    The only main considerations on this shell script are that you’ll want to stay with zsh and you’ll want to choose “as arguments” in the “Pass input” dropdown menu. Choosing “as arguments” allows you to take that file name and apply it to the For Loop in the shell script. I removed the echo "$f" because all it was doing was printing out the file name (which makes sense since it was the variable in this script).

    choose as arguments for the shell script, chris hoina, senior product manager, ords, oracle rest, database tools, sqlcl, sql developer, oracle rest
    Choosing “as arguments

    The Shell Script

    That cURL Command I copied from earlier looks like this:

    curl --location --request POST \
    --header "Content-Type: <CONTENT_TYPE>" \
    --data-binary @<FILE_NAME> \
    'https://abcdefghi1234567-darmok.adb.us-ashburn-1.oraclecloudapps.com/ords/admin/bank_transfers/batchload' 

    I made some modifications though. I made sure Content-Type was text/csv. And then I added some fancy options for additional information (more details on this here, go nuts) when I get a response from the database.

    REMINDER: I didn't know how to do this until about 30 mins before I got it to work. I'm emphasizing this because I want to drive home the point that with time and some trial-and-error, you too can get something like this to work!

    With my changes, the new cURL Command looks like this:

    curl -w 'Response Code: %{response_code}\nTotal Time: %{time_total} seconds\nUpload Speed: %{speed_upload} bytes/sec\nUpload Size: %{size_upload} bytes' --location --request POST \
    --header "Content-Type: text/csv" \
    --data-binary @"$f" \
    'https://abcdefghi1234567-darmok.adb.us-ashburn-1.oraclecloudapps.com/ords/admin/bank_transfers/batchload'

    What a mess…That -w option stands for write-out. When I receive the response from the Batch Load request, I’ll want the following information:

    • Response Code (e.g. like a 200 or 400)
    • Total Upload Time
    • Upload Speed
    • Upload Size

    All of that is completely optional. I just thought it would be neat to show it. Although, as you’ll see in a little bit, Apple notifications has some weird behavior at times so you don’t really get to see all of the output.

    I then applied the cURL command to the shell script, (with some slight modifications to the For Loop), and it ended up looking like this:

    new for loop with updated curl command, chris hoina, senior product manager, ords, oracle rest, database tools, sqlcl, sql developer, oracle rest
    New shells script with updated cURL command

    Here is what the output looked like when I did a test run (with a sample CSV):

    output of curl command, chris hoina, senior product manager, ords, oracle rest, database tools, sqlcl, sql developer, oracle rest
    Success on the cURL command

    Set Value of Variable

    All of that output, referred to as “Results”, will then be set as a variable. That variable will be henceforth known as the responseOutput (Fun fact: that is called Camel casing…I learned that like 3-4 months ago). You’ll first need to create the variable, and once you run the folder action, it’ll apply the results to that variable. Like this:

    Get Value of Variable and Display Notification

    Those next two modules simply “GET” that value of the variable/results and then sends that value to the Display Notification module. This section is unremarkable, moving on.

    And at this point, I was done. All I needed to do was save the script and then move on to the next step.

    Folder Actions Setup

    None of this will really work as intended until you perform one final step. I’ll right-click the target folder and select “Folder Actions Setup.” From there a dialog will appear; you’ll want to make sure both the folder and the script are checked.

    Trying it out

    Next, I emptied the folder. Then I dropped in a 5000-row CSV file and let Folder Actions do its thing. This entire process is quick! I’m loving the notification, but the “Show” button simply does not work (I think that is a macOS quirk though). However, when I go back to my Autonomous Database, I can 100% confirm that this ORDS Batch Load worked.

    Final thoughts

    This was relatively easy to do. In total, it took me about 3-4 days of research and trial and error to get this working. There is a lot I do not know about shell scripting. But even with a rudimentary understanding, you too can get this to work.

    Next, I’d like to create a dialog window for the notification (the output from the cURL Command). I believe you can do that in AppleScript; I just don’t know how yet.

    If you are reading this and can think of anything, please leave a message! If you want to try it out for yourself, I’ve shared the entire workbook on my GitHub repo; which can be found here.

    I’ll also be doing an extended video review of this, where I’ll recreate the entire automation from start to finish. Be on the lookout for that too!

  • Python and the Oracle Autonomous Database: Three Ways to Connect

    Python and the Oracle Autonomous Database: Three Ways to Connect

    Watch the deep dive videos:

    Part I

    Overview and connecting with the python-oracledb library

    Part II

    Connecting with Oracle REST APIs unauthenticated

    Part III

    Custom Oracle REST APIs with OAuth2.0 Authorization

    Welcome back

    I finally had a break in my PM duties to share a small afternoon project [I started a few weeks ago]. I challenged myself to a brief Python coding exercise. I wanted to develop some code that allowed me to connect to my Autonomous Database using either our python-oracledb driver (library) or with Oracle REST Data Services (ORDS).

    I undertook this effort as I also wanted to make some comparisons and maybe draw some conclusions from these different approaches.

    NOTE: If you don't feel like reading this drivel, you can jump straight to the repository where this code lives. It's all nicely commented and has everything you need to get it to work. You can check that out here. 

    The test files

    Reviewing the code, I’ve created three Python test files. test1.py relies on the python-oracledb library to connect to an Oracle Autonomous database while test2.py and test3.py rely on ORDS (test3.py uses OAuth2.0, but more on that later).

    Configuration

    Configuration directory

    I set up this configuration directory (config_dir) to abstract sensitive information from the test files. My ewallet.pem and tnsnames.ora files live in this config_dir. These are both required for Mutual TLS (mTLS) connection to an Oracle Autonomous database (you can find additional details on mTLS in the docs here).

    config directory files first test case, chris hoina, senior product manager, oracle rest apis, database tools
    ewallet.pem and tnsnames.ora files

    Other files

    oauth testyurls and wallet creds files for tests, chris hoina, senior product manager, ords, oracle rest apis, database tools, sqlcl
    OAuth2.0, Test URLs, and Wallet Credential files

    Other files include oauth2creds.py, testurls.py, and walletcredentials.py. Depending on the test case, I’ll use some or all of these files (you’ll see that shortly).

    NOTE: If not obvious to you, I wouldn't put any sensitive information into a public git repository. 

    Connecting with python-oracledb

    One approach to connecting via your Oracle database is with the python-oracledb driver (library). An Oracle team created this library (people much more experienced and wiser than me), and it makes connecting with Python possible.

    FYI: I’m connecting to my Autonomous Database. If you want to try this, refer to the documentation for using this library and the Autonomous database. You can find that here.

    The Python code that I came up with to make this work:

    #Connecting to an Oracle Autonomous Database using the Python-OracleDB driver.
    
    import oracledb
    
    # A separate python file I created and later import here. It contains my credentials, so as not to show them in this script here. 
    
    from walletcredentials import uname, pwd, cdir, wltloc, wltpwd, dsn
    
    # Requires a config directory with ewallet.pem and tnsnames.ora files.
    
    with oracledb.connect(user=uname, password=pwd, dsn=dsn, config_dir=cdir, wallet_location=wltloc, wallet_password=wltpwd) as connection:
        with connection.cursor() as cursor:
    
    # SQL statements should not contain a trailing semicolon (“;”) or forward slash (“/”).
    
            sql = """select * from BUSCONFIND where location='ZAF'
            order by value ASC """
            for r in cursor.execute(sql):
                print(r)

    In Line 7, you can see how I import the wallet credentials from the walletcredentials.py file. Without that information, this code wouldn’t work. I also import the database username, password, and configuration directory (which includes the ewallet.pem and tnsnames.ora files).

    From there, the code is pretty straightforward. However, some library-specific syntax is required (the complete details are in the docs, found here), but aside from that, nothing is too complicated. You’ll see the SQL statement in Lines 16-17; the proper SQL format looks like this:

    SELECT  * FROM busconfind WHERE location='zaf'
    ORDER BY value ASC;

    And here is an example of this SQL output in a SQL Worksheet (in Database Actions):

    testing sql in database actions for python-oracledb driver, chris hoina, senior product manager, oracle rest apis, ords, python, autonomous database
    Reviewing the SQL in Database Actions
    FYI: This is a Business Confidence Index data-set, in case you were curious (retrieved here). 

    That SQL allows me to filter on a Location and then return those results in ascending order according to the Value column. When I do this using the python-oracledb driver, I should expect to see the same results.

    NOTE: You've probably noticed that the SQL in the python file differs from that seen in the SQL Worksheet. That is because you need to escape the single quotes surrounding ZAF, as well as remove the trailing semi-colon in the SQL statement. Its all in the python-oracledb documentation, you just have to be aware of this.

    Once I have all the necessary information in my walletcredentials.py file, I can import that into the test1.py file and execute the code. I chose to run this in an Interactive Window (I’m using VS Code), but you can also do this in your Terminal. In the images (from left to right), you’ll see the test1.py file, then a summary of the output from that SQL query (contained in the test1.py code), and finally, the detailed output (in a text editor).

    Wrap-up

    For those that have an existing Free Tier tenancy, this could be a good option for you. Of course, you have to do some light administration. But if you have gone through the steps to create an Autonomous database in your cloud tenancy, you probably know where to look for the tnsnames.ora and other database wallet files.

    I’m not a developer, but I think it would be nice to simplify the business logic found in this Python code. Maybe better to abstract it completely. For prototyping an application (perhaps one that isn’t micro services-oriented, this could work) or for data- and business analysts, this could do the trick for you. In fact, the data is returned to you in rows of tuples; so turning this into a CSV or reading it into a data analysis library (such as pandas) should be fairly easy!

    Connecting via ORDS: sans OAuth2.0

    Auto-REST and cURL

    I’m still using the “devuser” (although this may be unnecessary, as any unsecured REST-enabled table would do). I’m using the same table as before; the only change I’ve made is to auto-REST enable the BUSCONFIND table for the test2.py code.

    In the following images, I’m retrieving the cURL command for performing a GET request on this table.

    NOTE: In a recent ORDS update, we made available different shell variations (this will depend on your OS); I've selected Bash.

    From there, I take the URI (learn more on URIs) portion of the cURL command and place it into my browser. Since this table is auto-REST enabled, I’ll only receive 25 rows from this table.

    NOTE: The ORDS default pagination is limit = 25.

    The code

    And the code for this test2.py looks like this:

    # Auto-REST enabled with ORDS; in an Oracle Autonomous Database with query parameters.
    
    import requests
    import pprint
    
    # Importing the base URI from this python file.
    
    from testurls import test2_url
    
    # An unprotected endpoint that has been "switched on" with the ORDS Auto-REST enable feature. 
    # Query parameters can be added/passed to the Base URI for GET-ing more discrete information.
    
    url = (test2_url + '?q={"location":"ZAF","value":{"$gt":100},"$orderby":{"value":"asc"}}}')
    
    # For prototyping an application, in its earlier stages, this could really work. On your front end, you 
    # expect the user to make certain selections, and you'll still pass those as parameters. 
    # But here, you do this as a query string. In later stages, you may want to streamline your application
    # code by placing all this into a PL/SQL or SQL statement. Thereby separating application
    # logic and business logic. You'll see this approach in the test3.py file. 
    # This works, but you can see how it gets verbose, quick. Its a great jumping-off point.
    
    responsefromadb = requests.get(url)
    pprint.pprint(responsefromadb.json())

    Lines 8 and 13 are the two areas to focus on in this example. In Line 8 imported my URL from the testurls.py file (again, abstracting it, so it’s not in the main body of the code).

    test2 python file and testurls file for second test, chris hoina, senior product manager, oracle rest apis, ords, python, autonomous database
    The test2.py and testurls.py files

    And then, in Line 13, I appended a query string to the end of that URL. ORDS expects the query parameters to be a JSON object with the following syntax:

    [ORDS Endpoint]/?q={"JSON Key": "JSON Value"}

    The new, complete query string below requests the same information as was requested in the test1.py example:

    url = (test2_url + '?q={"location":"ZAF","value":{"$gt":100},"$orderby":{"value":"asc"}}}')

    This string begins with that same BASE URI for the ORDS endpoint (the auto-REST enabled BUSCONFIND table) and then applies the query string prefix “?q=” followed by the following parameters:

    1. Filter by the location "ZAF"
    2. Limit the search of these locations to values (in the Value column) greater than ($gt) 100
    3. Return these results in ascending order (asc) of the Value column
    NOTE: You can manipulate the offsets and limits in the python-oracledb driver too. More info found here. And filtering in queries with ORDS can be found here. 

    And if I run the test2.py code in the VS Code Interactive Window, I’ll see the following summary output.

    summary output from python test2, chris hoina, senior product manager, oracle rest apis, ords, python, autonomous database
    Summary output from the response in test2.py

    Here is a more detailed view in the VS Code text editor:

    detailed output from python test2, chris hoina, senior product manager, oracle rest apis, ords, python, autonomous database
    Detailed output with helpful links

    Wrap-up

    A slightly different approach, right? The data is all there, similar to what you saw in the test1.py example. There are a few things to note, though:

    • The consumer of this ORDS REST API doesn’t need access to the database (i.e. you don’t need to be an admin or have a schema); you can perform GET requests on this URI.
    • The response body is in JSON (ubiquitous across the web and web applications)
      • Also, language and framework agnostic (the JSON can be consumed/used widely, and not just with Python)
    • You are provided a URI for each item (i.e. entry, row, etc.)
    • No need for SQL; just filter with the JSON query parameters
    • No business logic in the application code
    • Needless to say, no ORMs or database modeling is required for this approach

    However…security is, ahem…nonexistent. That is a problem and flies in the face of what we recommend in our ORDS Best Practices.

    Connecting via ORDS: secured with OAuth2

    Note: This is an abbreviated explanation, I'll be posting an expanded write-up on this example post haste!

    Since this is what I’m considering “advanced” (it’s not difficult, there are just many pieces) I’m going to keep this section brief. Long story short, I’ll take those query parameters from above and place them into what is referred to as a Resource Handler.

    TIME-OUT: Auto-REST enabling a database object (the BUSCONFIND table in this case) is simple in Database Actions. Its a simple left-click > REST-enable. You saw that in the previous example. You are provided an endpoint and you can use the query parameters (i.e. the JSON {key: value} pairs) to access whatever you need from that object. 
    
    However, creating a custom ORDS REST endpoint is a little different. First you create a Resource Module, next a (or many) Resource Template/s, and then a (or many) Resource Handler/s. In that Resource Handler, you'll find the related business logic code for that particular HTTP operation (the menu includes: GET, POST, PUT, and DELETE). 

    The Resource Module

    The process of creating a custom ORDS API might be difficult to visualize, so I’ll include the steps I took along with a sample query (in that Resource Handler) to help illustrate.

    Chances are you may be the administrator of your Always Free tenancy, so you have full control over this. Other times, you might be provided the REST endpoint. In that case, you may not ever have to worry about these steps. Either way, you can see how we’re simulating (as well as both abstracting and keeping the business logic in the database) the query with this final example (test3.py).

    Security

    The OAuth 2.0 authorization framework enables a third-party application to obtain limited access to an HTTP service, either on behalf of a resource owner by orchestrating an approval interaction between the resource owner and the HTTP service, or by allowing the third-party application to obtain access on its own behalf.

    RFC 6749: The OAuth 2.0 Authorization Framework

    I’ll keep this section brief, but I’m protecting this resource through the aid of an ORDS OAuth2.0 client. I’ve created one here:

    creating an oauth2 client for test3 example, chris hoina, senior product manager, oracle rest apis, ords, python, autonomous database
    After creating a client you can use the provided URL for requesting a new Bearer Token

    And, as you’ll see shortly, I’ll rely on some Python libraries for requesting an Authorization Token to use with the related Client ID and Client Secret. If you want to nerd out on the OAuth2.0 framework, I challenge you to read this.

    test3.py example

    NOTE: Remember, I'm keeping this section intentionally brief. It deserves a slightly deeper dive, and class is almost over (so I'm running out of time). 

    The code for this example:

    # Custom ORDS Module in an Oracle Autonomous Database.
    
    import requests
    from requests_oauthlib import OAuth2Session
    from oauthlib.oauth2 import BackendApplicationClient
    import pprint
    import json
    
    # Importing the base URI from this python file.
    
    from testurls import test3_url
    
    # A separate python file I created and later import here. It contains my credentials, 
    # so as not to show them in this script here. 
    
    from oauth2creds import token_url, client_id, client_secret
    
    token_url = token_url
    client_id = client_id
    client_secret = client_secret
     
    client = BackendApplicationClient(client_id=client_id)
    oauth = OAuth2Session(client=client)
    
    token = oauth.fetch_token(token_url, client_id=client_id, client_secret=client_secret)
    
    bearer_token = token['access_token']
    
    # Location can be anything from the table. Now, only the single variable needs to be passed. Business logic has been abstracted somewhat; as it now resides within
    # ORDS. This could make your application more portable (to other languages and frameworks, since there are fewer idiosyncracies and dependencies):
    
    location = "ZAF"
    
    # print(location)
    
    # ------------------------------------------------------------------------------ # 
    # In Database Actions, we:
    #   1. Create an API Module
    #   2. Then create a Resource Template
    #   3. Finally, a GET Resource Handler that consists of the code from test1.py:
    
    #           select * from BUSCONFIND where location= :id
    #               order by value ASC
    # ------------------------------------------------------------------------------ # 
    url = (test3_url + location)
    # print(url)
    
    responsefromadb = requests.get(url, headers={'Authorization': 'Bearer ' + bearer_token}).json()
    
    # This step isn't necessary; it simply prints out the JSON response object in a more readable format.
    
    pprint.pprint(responsefromadb)

    Lines 11 and 16 deserve some attention here. The URL for Line 11 comes from the testurls.py file; seen in the previous example. And the contents from Line 16 come from the oauth2creds.py file. Here are the files, side-by-side:

    test3 python oauthcreds and test url files, chris hoina, senior product manager, oracle rest apis, ords, python, autonomous database
    The test3.py, testurls.py, and oauth2creds.py files

    As you can see in the testurls.py file, I’m relying on the test3_url for this example. And the OAuth2.0 information you see comes directly from the OAuth Client I created in Database Actions:

    creating an oauth2 client for test3 example, chris hoina, senior product manager, oracle rest apis, ords, python, autonomous database
    In this image, you can see the Client ID and Client Secret

    If I put that all together, I can execute the code in test3.py and “pretty print” the response in my Interactive Window. But first I need to adjust the Resource Handler’s URI (the one I copied and pasted from the “REST Workshop”). It retains the “:id” bind parameter. But the way I have this Python code set up, I need to remove it. It ends up going from this:

    test3_url = 'https://test3_url = 'https://abc123def456ghi789-darmok.adb.us-ashburn-1.oraclecloudapps.com/ords/devuser/myapi/locations/:id'

    To this:

    test3_url = 'https://abc123def456ghi789-darmok.adb.us-ashburn-1.oraclecloudapps.com/ords/devuser/myapi/locations/'

    Otherwise, if I don’t remove that bind parameter when the code executes, the target URL will end up looking like this:

    'https://abc123def456ghi789-darmok.adb.us-ashburn-1.oraclecloudapps.com/ords/devuser/myapi/locations/:idZAF'

    With that out of the way, I can run this code and review the output.

    From top-to-bottom, left-to-right you’ll see I first execute the code in the Interactive Window. From there I can review a summary of the response to my GET request. That pretty print library allows us to see the JSON array in a more readable format (one that has indentation and nesting); which you can see in the second image. The third image is a more detailed view of the first half of this response. And I include the final image to highlight the helpful URLs that are included in the response body.

    Since I know my limit = 25, and the 'hasMore': True (seen in the output in that third image) exists, I know there are more items. You can adjust the limit and offset in subsequent requests, but I’ll save that for another day.

    Wrap-up

    You can probably tell, but this is like an expansion of the previous example. But instead of relying on the auto-REST enabling, you are in full control of the Resource Module. And while you don’t need to use OAuth2.0 it’s good practice to use it for database authentication. You can see how the response comes through a little differently, compared to the previous example, but still very similar.

    In this example, I did all the work, but that might not be the case for you; much of it might be handled for you. The main thing I like about this example is that we rely on stable and popular Python libraries: requests, requests_oauthlib, and oautlib.

    The fact that this is delivered as a JSON object is helpful as well (for the same reasons mentioned in the second example). And finally, I enjoy the fact that you only need to pass a single parameter from your (assumed) presentation layer to your application layer; an example might be a selection from an HTML form or drop-down menu item.

    The end

    We’re at the end of this fun little exercise. As I mentioned before, I will expand on this third example. There are so many steps, and I think it would be helpful for people to see a more detailed walk-through.

    And be on the lookout (BOLO) for a video. There’s no way around this, but a video needs to accompany this post.

    And finally, you can find all the code I review in this post in my new “blogs” repository on GitHub. I encourage you to clone, fork, spoon, ladle, knife, etc…

    Oh and…