Thor/Merlin 4.5: Datatree merging and garbage collection

Datatree merging

In Thor databases, data is added to existing datatrees by "tree merging" via the dt_thor_tdtmerge() method. The implementation of this method has changed in 4.51. Except for providing higher performance, this change should be invisible to users and programmers.

Old method

Previously, tree-merging was done by the client process rather than the thorserver. (Thor toolkit programmers didn't have to be aware of this, since the toolkit takes care of all such things.) Also, merged datatrees were always written to the end of the database file, leaving "holes" in the file which were recovered by "crunching". The advantage of this method was that it allowed the thorserver to be very simple (thorserver didn't have to know very much about datatrees). The disadvantage of this method is that it is slow: it requires three network transactions to perform a merge (~15 mS). The old method is illustrated below:

New method

In Release 4.5, we are introducing a "smarter" thorserver which is able to merge datatrees locally. The new method for datatree merging allows the client to pass datatrees to be merged to the server which then handles the rest of the task locally. This allows datatree merging to be done with a single network transaction.

The 4.5 thorserver is also "smarter" about replacing datatrees in database files: datatrees are replaced "in-place" if they'll fit, reducing the amount of automatic database crunching required (which is another slow step). To make this more effective, extra reserved space is inserted following commonly-used cross-references in an adaptive manner. Such space is not subject to garbage collection (except when the database is explicitly crunched).

This change in a fundamental Thor method is entirely invisible to users and programmers alike. Such behavior is a consequence of the "transparent object interface" used by the Daylight Toolkit. (Ya gotta love it, ain't OOP grand?)

Daylight Chemical Information Systems, Inc.