Attunity MS-SQL CDC - transient storage file structure

MS-SQL CDC transient storage is maintained in order to preserve the availability of logged data which is restricted due to periodical MSSQLserver truncations.

The general implementation characteristics of the MS-SQL CDC transient storage package are:
v Transient storage resides at a dedicated working folder, associated with a single LGR service / database instance.
v Total allocated storage size is bounded.
v Storage units (=files) are each of limited size.
v Files are organized as plain flat ‘sequential variable’ files.
v Transient storage package is equipped with means for a simple fast access to a given ‘LSN keyed’ entry from which sequential access would start – as this is the basic mind of the “CDC’ access mode.
v A maintenance policy is applied by the LGR service whenever allocated space is about to exhaust, using the internal cleanup functionality.
v Accessing transient storage files is shared among LGR as a (single) writer and AGNET(s) as (multiple) readers.

When looking at a transient storage folder – following is seen:

09/04/2006 04:32 PM <DIR> .
09/04/2006 04:32 PM <DIR> ..
09/04/2006 03:32 PM 500,135 00002a73_00000273_0025.cdc
09/04/2006 03:32 PM 0 00002a75_0000010a_0071-00002a77_000000f6_0013.toc
09/04/2006 03:32 PM 500,088 00002a75_0000010a_0071.cdc
09/04/2006 03:32 PM 0 00002a77_000000f6_0014-00002a78_00000161_0001.toc
09/04/2006 03:35 PM 306,183 00002a77_000000f6_0014.cdc
09/04/2006 04:32 PM 2,591 00002a78_00000164_0002.cdc
09/04/2006 04:32 PM 0 00002a78_00000164_0002.toc
26 File(s) 4,824,771 bytes
2 Dir(s) 43,412,848,640 bytes free>

The *.CDC files actually contains the captured data. They are named after the first LSN key stored in them – a naming convention adopted for providing uniqueness.
For each *.CDC file there is a *.TOC companion empty file. TOC files – altogether – are an ‘on disk Table of Contents’ (=TOC) of the transient storage as a whole, implemented as directory structure entries.
Each TOC entry is named after the lowest LSN and the Highest LSN held in its associated CDC file.
Locating a specific LSN key for applying CDC stream positioning is based on directory scanning for the relevant *.TOC file.
Other transient storage activities – cleanup, sealing (=COMMIT), recovery (all which are discussed in a separate post) – are also based on native directory operations.