Confocor3 Raw Data Specification efg, 2 May 2007 Stowers Institute for Medical Research Klaus Weisshart from Zeiss provided this updated annotated hex dump of a ConfoCor3 file (in a 5/2/2007 8:17 E-mail): The ConfoCor3 Operating Manual (Release 4.0, March 2006) provided some additional details about the fields, but the file layout has changed somewhat. Using this new information, I now interpret a raw Confocor3 file as follows: ConfoCor3RawDataSpec.doc 2 May 2007 1 Confocor3 Raw Data Specification ConfoCor 3 File Layout Example filename: dc0a40540b831f7efb272a095c923f5_R10_P1_K1_Ch1.raw Field File Identifier Start Size Type [Bytes] 0 64 Character[64] Comments File Identifier with channel number. This field can be parsed into several parts: Measurement Identifier 64 16 Integer4[4] Measurement Position Kinetic Index 80 4 Integer4 84 4 Integer4 Repetition Number Sampling Frequency 88 4 Integer4 92 4 Integer4 Reserved Pulse Distances Carl Zeiss ConfoCor3 raw data file version 3.000 Channel 1 This will be assigned to all channels of the same repetition. Hence, in a cross-correlation experiment, the two auto-correlation pairs that belong together can be identified. In this example: dc0a40540b831f7efb272a095c923f5 (Note: this is the hex form from 4 32-bit little endian integers processed one at a time) Zero based: P1 in filename means this will be 0 Zero based: K1 in filename means this will be 0 Zero based: R10 in filename means this will be 9 96 128 32 Integer4[8] Integer4[ Filesize – 128 0x002D3101 (little endian) 20000000 (decimal) (i.e., 20 MHz) Always 0s? (Filesize -128)/4] Interger4 = 4-byte "little endian" unsigned integer The Time Delta Array (Pulse Distances) are all Integer4s representing detector clocks at the given Sampling Frequency (usually 20 MHz). The size of this array can be computed: [FileSize – 128] / 4 = number of elements in time delta array. ConfoCor3RawDataSpec.doc 2 May 2007 2 Confocor3 Raw Data Specification With a 20 MHz Sampling Frequency, the sum of the time deltas will overflow a 32-bit integer after 214.74 seconds of data collection. If data samples are shorter than this, then 32-bit integers are sufficient to process the time delta array. Example of Confocor3 Processing Form Time Array Convert time deltas to absolute time values (detector clocks): ΔT[Clocks] = 484459 261406 32838 15657 23050 (decimal) T0[Clocks] = 0 T1 = T0 + ΔT0 = 0 + 484459 = 484459 T2 = T1 + ΔT1 = 484459 + 261406 = 745865 T3 = T2 + ΔT2 = 745865 + 32838 = 778703 ... T = 484459 745865 778703 794360 817410 This array of time values, T, in detector clocks can be divided by the sampling frequency, 20 MHz (i.e., 20E6) to give values in seconds. Use Integer Division to Bin Bins should be chosen to correspond to a time interval that is a multiple of the sampling frequency. Each "width" of the bin in bits can be chosen to be any number, say BinSize. Simple integer division (DIV) gives the bin index, and the counts in each bin can be easily computed: BinIndex = T DIV BinSize Bins[BinIndex]++ [sum bin counts in IDL] ConfoCor3RawDataSpec.doc 2 May 2007 3 Confocor3 Raw Data Specification Annotated Example: Filename: dc0a40540b831f7efb272a095c923f5_R10_P1_K1_Ch1.raw <------------ hex -------------> <---- ASCII ---> File identifier (including channel) [first 64 bytes] 1 4361726C205A6569737320436F6E666F 2 436F7233202D20726177206461746120 3 66696C65202D2076657273696F6E2033 4 2E303030202D204368616E6E656C2031 Carl Cor3 file .000 Zeiss Confo - raw data - version 3 - Channel 1 Measurement identifier [16 bytes (4 4-byte integers)] 5 05A4C00DF731B840A072B2EFF523C995 .....1.@.r...#.. | | | | | 0DC0A40540B831F7EFB272A095C923F5 (big endian) Note: Zeiss should have processed this byte-by-byte so a hex dump matches the value in the filename. The "big endian" format is used in the filename, but with any leading zero dropped. Various parameters Detector clock frequency (Hz) 6 000000000000000009000000002D3101 .............-1. Position (zero based): 00000000 Kinetics index (zero based): 00000000 Repetition (zero based): 09000000 little endian → 00000009 big endian → 9 (decimal) Detector clock frequency: 0x002D3101 (little endian) = 0x01312D00 (big endian) = 20000000 (decimal) Ignore 32 bytes 7 00000000000000000000000000000000 ................ 8 00000000000000000000000000000000 ................ Skip 8 4-byte integers Array of pulse distances (detector clocks) 9 9F000000EA0000007D00000089000000 ........}....... 10 C10000001B0000002802000078020000 ........(...x... 11 100000007D000000B800000092000000 ....}........... . . . 13243 62010000370000000F0400006C010000 b...7.......l... 13244 59010000470100005A040000A3020000 Y...G...Z....... 13245 21010000390000000B070000 !...9....... Number of elements in array: (FileSize – 128) / 4 In this case: (211,916 -128)/4 = 52947 events The first event delta is represented in "little endian" form as 9F000000, which would be 0x0000009F in "big endian", which is the decimal number 159. The last event is 0B070000 "little endian". ConfoCor3RawDataSpec.doc 2 May 2007 4 Confocor3 Raw Data Specification IDL Example 1: FCS3Demo1.prj Project IDL Structure: PRO TConfoCor3Header__define struct = {TConfoCor3Header, FFileIdentifier : FMeasurementIndentifier: FMeasurmentPosition : FKineticIndex : FRepetitionNumber : FSamplingFrequency : FReserved : } $ $ bytarr(64), ulonarr(4), 0ul, 0ul, 0ul, 0ul, ulonarr(8) $ $ $ $ $ $ $ ; zero based ; zero based ; zero based ; 20,000,000 (0x002D3101 little endian) ; normally 00000000s? END Output from IDL FCSDemo1.pro program file in U:\efg\lab\IDL\FCS3 that processed file dc0a40540b831f7efb272a095c923f5_R10_P1_K1_Ch1.raw Carl Zeiss ConfoCor3 - raw data file - version 3.000 - Channel 1 Channel 1 Measurement Identifier = hex: DC0A40540B831F7EFB272A095C923F5 Measurement Position = hex:00000000, dec: 0 Kinetic Index = hex:00000000, dec: 0 Repetition Number = hex:00000009, dec: 9 Sampling Frequency = 20000000 Reserved = hex: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 First/Last Event Deltas (hex/dec) = 0000009F 159 0000070B 1803 ConfoCor3RawDataSpec.doc 2 May 2007 5 Confocor3 Raw Data Specification IDL Example 2: FCS3Demo2.prj Project Output from IDL FCSDemo2.pro program file in U:\efg\lab\IDL\FCS3 that processed file U:\efg\lab\IDL\FCS3\R1_P1_K1_Ch2.raw bits in each bin = 40800 for bin width(sec) = 0.00204000 This matches Delphi FCS Viewer program for the same dataset. ConfoCor3RawDataSpec.doc 2 May 2007 6