MUG '01 - Weininger - http toolkit

MUG '01 -- 15th Daylight User Group Meeting -- 6 - 9 Mar 2001

HTTP toolkit

Dave Weininger, Daylight CIS

1. HTTP service as a program paradigm

Requirements
- Full-strength delivery of GUI to all kinds of clients
- Non-GUI application programs would be nice, too.
- Network orientation for integration as a tool
- Administration-free program delivery
- Secure operations in a (mainly) stateless environment
- Scale from one machine to LAN to enterprise to www
- Robust. Reliable. Testable. Supportable.

2. HTTP service as a program paradigm

Compelling motivations
- MIT-X/XView limited future, limited audience
- DCGI right direction but not supportable in long run
- Current information delivery is limited by interfaces
- "Bite the bullet and do it right" is a long term winner
- 100% backward compatibility with DCGI is a bonus

3. HTTP toolkit

HTTP object
- An HTTP object represents an HTTP service
- dt_http_alloc() creates one
- dt_http_loop() starts it
- dt_dealloc() destroys it
- Everything else is controlled by object properties

4. HTTP toolkit

HTTP object properties
- Factory properties are in the "_" namespace
- Very few properties needed for basic operation
  - "_path" -- request path
  - "_reply" -- reply content
  - "_mime" -- reply type
- Large number of properties allow complete control
- Object-property API is likely basis of Daylight 5.x

5. New concepts

Daylight methods
The HTTP toolkit requires the introduction of object methods to the Daylight toolkit API. In the HTTP toolkit these serve as callbacks, but methods may be used for many other purposes also. In its current implementation, the HTTP toolkit defines five such methods as HTTP object properties (_init, _loop, _beat, _error, _exit). Of these, only one (_loop) is needed for basic operation.
Service domain
Something that ought to be part of the HTTP protocol, but it doesn't seem to be, so it was invented as part of the HTTP toolkit. A service domain is simply a set of IPs which an HTTP server will service. It may be a network or a list of IPs. The default service domain is the server's own (A-, B-, or C-class) network, but it can be set like any other HTTP object property to, for instance, a subnet or www. HTTP requests originating from outside the service domain get a 403 Forbidden response.

6. `/factory` services

The HTTP toolkit provides a set of built-in resources, to enhance robustness and convenience. The caller's _loop method is invoked for /factory URI's; such requests can be denied (return 403 Forbidden or 404 Not Found) use the intrinsic service (return 303 Moved Temporarily), or serviced in the normal manner (overriding intrinsic behavior).

/factory/etc/
current status and object property information
/factory/help/
documentation about HTTP toolkit and server usage
/factory/image/
static images such as the Daylight sunburst logo
/factory/java/ and /factory/javascript/
.jar, Class and .js resources

7. Other cool stuff

Forms
Data from posted forms are available as a sequence of name-value pairs as the _posts request property.
File upload
Multipart encoding is supported to allow file upload from the client's machine. Data is provided as a normal posted property _value. Additionally, the "_filename" property is defined.
Cookies
Netscape cookies are supported to provide a limited amount of state information. Default or user-provided encryption is supported.
Authorization
HTTP 2.0 challenge/authentication protocol is supported.

7. dayfetch

dayfetch is a simple program which accepts a URL/URI with a local filename argument, makes the HTTP request, then returns the content to a file or standard output. Combined with HTTP servers which provide text/plain services, this can provide the functionality of serial programs such as thorfilters, clogp, mol2smi, etc.

8. Outstanding issues

HTTP applications as DCGI replacements
The first use of the HTTP toolkit will be to create Daylight application programs. We can provide a complete replacement for DCGI (something we've never fully supported due to it's shortcomings). The HTTP-toolkit based DCGI replacement should be superior to in performance, robustness and supportability. Is everyone OK with a straight replacement strategy?
HTTP toolkit as product
The HTTP toolkit is designed with the idea that the toolkit itself will be a product, much like the other Daylight toolkits. It's not tremendously chemistry-oriented though ... should we release/support it as mainstream Daylight software? If so, should we do so right away (i.e., with the first release of Daylight HTTP-oriented servers)?
Where do we draw the line?
It's tempting to build a bunch of chemistry-oriented utilities right into the HTTP toolkit. For instance, it's trivial to deliver Grins.jar which provides robust JavaGrins access to all programs using the HTTP toolkit. What makes sense to build-in, what to make a separate service? File format conversion? SMILES canonicalization? Eventually it'll grow fat. On the other hand, it all lives in a shared runtime library. Humm.

9. Thanks to

Roger Sayle for inspiration and armor