StackHub Bulletin BoardAdvertise your announcements here!
API

afHttpClientExt

Axon functions for making HTTP requests to REST APIs
afHttpClientExt

Registered StackHub users may elect to receive email notifications whenever a new package version is released or a comment is posted on the forum.

There are 0 watchers.

v2.0.2

HTTP Client Ext is an extension for SkySpark v3.0.18+ that provides powerful Axon functions for making HTTP requests and calling REST APIs.

HTTP Client Ext boasts the following features:

  • support for all HTTP methods, GET, POST, PUT, DELETE, OPTIONS, etc...
  • support for custom HTTP headers
  • posting forms / JSON / XML
  • upload files and multi-part forms
  • built in authentication for BASIC, DIGEST, and Project Haystack / SkySpark SCRAM
  • gzip and deflate response de-compression
  • redirect following
  • session cookies

HTTP Client Ext v2 has a clean and simple API for building REST requests, it maintains session state including cookies, and has built in extensible support for authentication schemes.

HTTP Client Ext is free to download and evaluate but a licence must be purchased for production use. See Licensing for details.

Axon Functions

HTTP Client Ext provides the following Axon methods:

The simplest function to retrieve the contents of a URL would be:

afHttpGet(`https://www.fantomfactory.com/`)

Beyond this, HTTP Client Ext maintains the notion of a session for the duration of the Axon context it runs it. (The scope of the Axon context is usually the top-level function being run.) During this session, cookies are saved and automatically resent, as are any authentication credentials.

While this works well inside custom Axon functions, it can pose problems when trying out HTTP calls in the standard Tools App - because in the Tools app, the session ends after each command.

To get around this, wrap your calls to HTTP Client Ext in a mini-function. For example, to return the HTTP response for display in the tools app, you could call:

(()=>do afHttpGet(`https://www.fantomfactory.com/`); afHttpResponse({safeNames}); end)()

Body sniffing

HTTP Client Ext makes it easy to send different types of content by passing different objects as the body.

The body is inspected (sniffed) and handled appropriately, as outlined below. HTTP Client Ext takes care of all the intricate technical stuff like setting Content-Type and Content-Length request headers, so you don't have to.

(Note to override any default Content-Type header, just supply your own in the headers Dict.)

Str to XML / JSON / Plain Text

If body is a Str then it is sent (as UTF-8 encoded characters):

If the string starts with < and ends with > then it is taken to be XML and the content type set to text/xml.

If the string starts with { and ends with }, or if it starts with [ and ends with ] then it is taken to be JSON and the content type set to application/json.

All other strings are taken to be plain text and the content type set to text/plain.

afHttpPost(`http://example.com`, null, """{"name":"Steve", "beer":"London Pride"}""")

Grid to Zinc

If the body is a Grid then it is written out in zinc format and the content type set to text/zinc.

afHttpPost(`http://example.com`, null, `io/upload.zip`)

Dict to Form

If the body is a Dict then it is taken to be form values and written out appropriately. The content type is set to application/x-www-form-urlencoded.

This mimics submitting a form on a web page.

afHttpPost(`http://example.com`, null, {name:"Emma", score:11})

Uri to File

If the body is a Uri and starts with io/ then it is taken to represent a file in the project's io/ directory. The file is then read and its binary content piped out as the request body. The file's extension is used to set the content type.

Note that most file uploads are performed via multipart-forms, so try that before this.

afHttpPost(`http://example.com`, null, `io/upload.zip`)

Fn to Multipart Form

Axon Funcs are used to send multipart forms, which are usually used to upload files.

afHttpPost(`http://example.com`, null) (addPart) => do
    addPart("name", "Steve")
    addPart("file", `io/upload.zip`)
end

addPart() takes a name and a value, where name is the name of the form entry, and value may be any Grid, Str, or Uri object which is sniffed just like the body object.

REST APIs

The standard Axon ioReadJson() and Fantom Factory's afHttpWriteToJsonStr() methods can be used make calls to REST APIs.

// a sample object structure
dict : {name:"Emma", sex:"female", score:11, likes:["Cakes","Adventure"]}

// convert the object to a JSON string
json : dict.afHttpWriteToJsonStr({prettyPrint, maxWidth:20})

// most REST APIs require some kind of authentication
afHttpAuth("bearer", "XXXX-XXXX")

// make the call
res  : afHttpPost(`http://example.com/`, null, json)

// convert the returned JSON string back into Axon objects
// use 'safeNames' if you want to display data in the Tools app
data : res.ioReadJson({safeNames})

Don't forget that PUT, DELETE, OPTIONS, and any other HTTP method may be invoked using the afHttpRequest() function.

Authentication

HTTP Client Ext comes with default implementations of BASIC, BEARER, DIGEST, and Project Haystack SCRAM authentication schemes. To use, call afHttpAuth() at the start of your session and the authentication mechanism will be played out on the next HTTP request.

Most schemes have a mechanism to resend an authentication token (such as a BEARER token) so you should only need to set the credentials once in the current session.

afHttpAuth("basic", "user", "secret")
afHttpGet(`http://example.com/secret1.txt`)
afHttpGet(`http://example.com/secret2.txt`)

Valid / supported authentication schemes to pass to afHttpAuth:

Keep passwords safe

Be safe, don't store passwords in code or plain text!

Use the core Axon function passwordSet() to securely set your password in the current project's password store. passwordSet() may be called manually at any point in time using a known Ref:

passwordSet(@xxxx-xxxx, "secretPassword")

Then, in your HTTP code, pass the same Ref to afHttpAuth and the password will be retrieved from the project's password store:

afHttpAuth("basic", "user", @xxxx-xxxx)
afHttpGet(`http://example.com/secret.txt`)

This keeps secrets safe from prying eyes!

Custom Authentication

Many REST APIs have a custom authentication scheme requiring bespoke HTTP calls before acquiring some sort of BEARER token or login cookie. It should be possible to make these HTTP calls with HTTP Client Ext, and any subsequent tokens can be set in the HTTP header using the usual afHttpGet and afHttpPost methods. But doing so leaves you in the sticky situation of having to look after your passwords in plain text.

Why not ask Fantom Factory to write the authentication for you?

We can create a mini SkySpark extension that adds a custom authentication just for you. That way you can keep your passwords safe in the SkySpark password store, and know the authentication has been programmed to the highest possible standards.

Contact Fantom Factory at ---------------------- for details.

HTTP Errors

Not every HTTP request returns as expected, sometimes the REST API server returns an error in the form of a HTTP Response Status Code.

A common code is 404 - Not Found but there are many more. A full list of status codes may be found on wikipedia.

When HTTP Client Ext encounters an error status code, generally a 4xx or 5xx code, then it throws an error. If you want to handle these error cases yourself, you can perform a standard try / catch and check the returned status code directly in the err.

try do
    afHttpGet(`https://fantom.org/error404`)

catch (err) do
    if (err->statusCode == 404)
        echo("Page not found!")
    else
        // perpetuate all other errors
        throw err
end

An alternative is to clear the default HTTP response handler for that particular status, then manually check the returned code yourself:

afHttpClear({statusCode:404})
afHttpGet(`https://fantom.org/error404`)

res : afHttpResponse()
if (res->statusCode == 404)
    echo("Page not found!")

Note that multiple HTTP status code handlers may be cleared at once using an x or * wildcard. For example, to stop following ALL redirects:

// disable redirect following
afHttpClear({statusCode:"30x"})

Debugging

It can be very useful to see exactly what HTTP traffic is being sent and received. Calling afHttpDebug() will turn on debugging; but just for the current session (in case you forget to turn it off again!).

With debugging on, HTTP requests and responses will be written to the console, giving full visibility as what's happening behind the scenes.

<afHttpClient.demo> {ext} [debug]

  HTTP Request:
  GET / HTTP/1.1
  Host: www.google.com
  User-Agent: afHttpClient/0.0.5 (Fantom/1.0.73.21 win32-x86_64)

<afHttpClient.demo> {ext} [debug]

  HTTP Response:
  HTTP/1.1 200 OK
  Date: Fri, 22 Nov 2019 10:44:12 GMT
  Expires: -1
  Cache-Control: private, max-age=0
  Content-Type: text/html; charset=ISO-8859-1
  P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
  Content-Encoding: gzip
  Server: gws
  Content-Length: 5418
  X-XSS-Protection: 0
  X-Frame-Options: SAMEORIGIN
  Set-Cookie: 1P_JAR=2019-11-22-10; path=/; domain=.google.com

  <!doctype html>
  <html lang="en-GB">
    <head>
      <meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
      <title>Google</title>
      ...
      ...
      ...
  </html>

Note that afHttpDebug() is restricted to super users only - you don't want any old user checking out those authentication requests!

To keep debugging turned on for longer, you can set the log level for HTTP Client Ext in the standard SkySpark Debug -> Log view.

SSL Handshake Exceptions

Should any of the HTTP functions throw an javax.net.ssl.SSLHandshakeException, then that signifies a problem with the underlying Java platform and not the HTTP Axon functions.

Note that SSL is not a type of encryption, it's an ever growing suite of standards and technology. Meaning web clients (in our case Java) has to continually grow to support it all.

Make sure Java supports the required server cipher by inspecting the server's SSL certificate in a browser, or by visiting SSL Labs. Then visit Oracle's Java Security Providers to ensure the cipher is supported. See StackOverflow's Problems connecting to HTTPS/SSL with Java client for details.

Note that strong encryption ciphers are often disabled in Java by default. To enable them, you may need to install Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files.

Licensing

HTTP Client Ext is a commercial product and requires a licence to be purchased to run in production.

Licences for HTTP Client Ext are tied to an SkySpark installation / host ID. This means to use HTTP Client Ext on multiple production servers will require multiple licences; one per server. You will need to enter your SkySpark licence ID when purchasing, see Where is my SkyArc Installation ID? for details.

Purchased licences are available from the My Licences page and should be downloaded to the /var/lic/ directory of your SkySpark installation.

Minor updates to HTTP Client Ext may be provided free of charge, but major updates will require a new licence to be purchased.

For support and comments, please email ----------------------.

Evaluation Mode

If HTTP Client Ext does not find a valid licence, it enters into an evaluation mode whereby a maximum of 20 HTTP calls may be made; after which HTTP Client Ext will throw a licence fault.

Evaluation mode is designed for developers to try out various features of HTTP Client Ext to assess its suitability for purpose.

Special Mentions

Special thanks go to Jaap Balvers and Pieter van der Mijle of BAM Energy Systems for sponsoring, supporting, and beta testing HTTP Ext 1.0.

Thanks also goes to Ian Habermann of AFM for (indirectly) sponsoring the DIGEST authentication scheme, and to Stehpen Frank of NREL for sound boarding the new v2.0 authentication features.

Published by Fantom Factory

Products & Services by Fantom Factory

Packages by Fantom Factory

Commercial packages

Free packages

Pricing options
HTTP Client Ext$195.00
Log in to purchase Download
Package details
Version2.0.2
LicenseCommercial
Build date2 months ago
on Sat 11th Jan
Requirements SkySpark v3.0.18
Depends on
File nameafHttpClientExt.pod
File size131.76 kB
MD55416e246e063efddc45e4bdf804c1777
SHA1 8bc134894c53c04b579ac93143473b2099a35bb0
Published by
Fantom FactoryDownload now
Also available via SkyArc Install Manager
Tags
Fantom Pod
Sky Arc Ext
Axon Funcs