Andor’s sCMOS camera solutions are capable of data rates that are markedly faster than other scientific camera technologies on the market. It is important to ensure that both the camera to PC interface, and the speed and capacity of the PC memory being used support the data output from the camera. The pixel readout speed that yields the fastest frame rate of 100 fps (full frame) relates to a data rate of ~ 850 MB/sec (single amplifier mode).
1.0 Optimizing data spooling: Identifying the ‘bottlenecks’
Some applications require fast kinetic series acquisitions that are sustainable for a relatively long duration. In such cases, the data must be spooled continuously to either a suitable fast PC hard drive solution or PC RAM, each with sufficient storage capacity.
The data transfer rates achievable over more extended kinetic series are limited either by:
The data bandwidth of the Camera Link/USB 3.0 interface interface between camera and PC or
The write speed of the hard drive (if spooling to hard drive is selected)
The maximum sustained speeds are ultimately limited by the interface bandwidth. In addition, time is taken for a ‘read request’ to be sent by the software to retrieve the next image block.
1.1 Single Camera Link Interface (‘3-tap’)
The single Camera Link interface (‘3-tap’) has a bandwidth limitation of ~ 250MB/sec, which translates to ~ 30 frames per second (fps) of 5.5 MP image size, single/dual amplifier mode.
1.2 Dual Camera Link Interface (‘10-tap’)
The dual Camera Link interface (‘10-tap’) has a bandwidth limitation of 850 MB/sec, translating to the full 100 fps in single amplifier mode. Thus, to achieve maximum available sustained speeds, a PC configuration should be capable of writing/spooling data at faster than this rate (see Section 4.0).
The maximum length of a kinetic series is determined by the capacity of PC RAM or hard drive that is assigned for spooling. The issue of determining achievable speeds is further compounded by the fact that data rates are also adjusted by user selected variables such as exposure time, pixel readout speed, ROI size, hardware binning or single/dual amplifier dynamic range modes.
1.3 USB 3.0 interface
The USB 3.0 interface has a bandwidth limitation of 335 MB/sec, which translates to 40 fps of 5.5 MP image size and 53 fps of 4.2 MP image size, single amplifier mode.
Figure 1 - The Dataflow monitor has raised a warning against the requested kinetic series. In this case the data rate exceeds that of the hard drive write speed. Options to rectify this include: (a) reduce frame rate / lengthen exposure time (b) maintain frame rate, but reduce ROI size (otherwise the frame rate increases and the data rate remains the same) (c) use hardware binning (d) use single amplifier mode (d) reduce kinetic series length to be within the 4 GB on-head camera memory of the Neo 5.5 sCMOS (e) spool to PC RAM (if greater than 4 GB in the case of the Neo 5.5 sCMOS).
2.0 Andor Dataflow Monitor
Andor developed the Dataflow monitor for the Neo 5.5 sCMOS and Zyla 5.5 & 4.2 sCMOS cameras to identify any system bottlenecks for a requested kinetic series. The Dataflow Monitor is accessible through the set-up dialogue of the Andor Solis acquisition software. This provides ‘up-front’ estimation as to whether the kinetic series conditions that have been requested by the user are able to support the data transfer and write bandwidths available from Camera Link/USB 3.0 interface and PC hard drive respectively.
The Dataflow monitor also estimates if the available storage capacity of camera, PC RAM or hard drive is sufficient for the requested acquisition. This informs the user of any potential data speed or capacity issues in advance of beginning an acquisition. A scenario in which the Dataflow monitor has accepted and flagged a kinetic series request, respectively is shown in Figure 1.
3.0 Exploiting the on-head memory buffer of the Neo 5.5 sCMOS
The Neo 5.5 sCMOS camera has a 4GB on-head memory buffer. This buffer can be used for a ‘data burst’ up to the maximum frame rate of the camera until it is full. Therefore a much higher frame rate can be attained during this time than can be supported over a more prolonged time. Once the onhead memory is full the frame rate will be limited by the Camera Link interface bandwidth, the hard drive write speed (if hard disk spooling is selected) and the other ‘processing’ overheads associated with the acquisition software.
In practice, one might expect a capacity of between 450 to 500 frames when operating at 100 fps, 5.5 megapixel (full resolution) in single amplifier mode (12-bit) i.e. the Neo on-head memory buffer can hold 4 to 5 seconds worth of kinetic series data under maximum frame rate conditions of the camera.
Using smaller ROI sizes to achieve the highest frame rates
It is also possible to use smaller ROI sizes and the Neo 5.5 on head memory buffer to achieve exceptionally high frame rates. For example:
Maintain frame rate and extend kinetic series length beyond 5 seconds, or
Achieve faster frame rates but stay within the 4 to 5 sec threshold
The Neo 5.5 sCMOS memory buffer can therefore be used to achieve high acquisition rates from even modestly specified PC configurations. For example, a hard drive may only be capable of writing data at 25 fps, yet a speed of 40 fps is required for a particular experiment. In this instance, it is still possible to acquire at 40 fps, the memory buffer filling at a rate of 15 fps. Under these circumstances, the kinetic acquisition could be run for up to ~ 30 seconds before the buffer becomes filled.
4.0 PC Recommendations - Speed tests
4.1 Zyla ‘10-tap’ kinetic series tests
PC solutions for Zyla 5.5 and Zyla 4.2 sCMOS that were tested by Andor over extensive kinetic series lengths are shown in Table 1. The dual Camera Link ‘10-tap’ configuration of the Zyla represents the option that is chosen for the fastest possible frame rate performance.
The test utilised Andor Solis acquisition software, internally triggered, rolling shutter and global shutter mode and 200 MHz/560MHz pixel readout speed, in both 12-bit (single amplifier) and 16-bit (dual amplifier) data range configurations.
The results indicate the sustainable frame rates, limited only by the capacity of hard drive or PC RAM. The PC configuration is based on the Dell T7910 with 2.6 GHz Eight Core and the Dell T5810 with 3.5 GHz Quad Core.
Speed tests used extensive kinetic series lengths which filled the 1 TB capacity of the SSD
2560 x 2160 – 120,000
2064 x 2048 – 160,000 frames
1980 x 1080 – 340,000 frames
512 x 512 – 2,600,000 frames
128 x 128 – 40,000,000 frames
System A utilizes 4 x 250 GB Solid State Drives (SSD) configured in RAID 0 for data spooling. An additional fifth hard drive is assigned to the operating system (Windows 7, 64-bit). (see Figure 3).
System B utilizes 64 GB RAM (4 x 16 GB) for direct data spooling. Solis acquisition software is a 64-bit application and thus allows the user to utilise the full RAM capacity. It is important to note that spooling to RAM does not allow for permanent storage and the user would need to send the data to a hard drive with equivalent capacity following data acquisition in order to save the kinetic series.
Table 1: Frame rates achieved by Zyla 5.5 and the Zyla 4.2 sCMOS ’10-tap’ on the Dell T7910 with 4x SSD (1 TB) in RAID 0 configuration (System A) and the Dell T5810 with 64 GB RAM (System B) and tested over extended kinetic series to fill the capacity of the SSD (1 TB) and the RAM (64 GB)
* System B with 64 GB RAM can acquire 6000 full frame (2560 x 2160 for Zyla 5.5 and 2048 x 2048 for Zyla 4.2) 12-bit images before the full capacity is utilised. For smaller ROI’s this limit is significantly reduced.
In consideration of these results:
System A has more capacity compared to the RAM solution in System B
Speeds attained from System B (RAM) can be considered to approximate the maximum sustained frame rate possible for each ROI size.
Performance achieved for System A was limited more by the write capacity of the hard drive configuration.
From an SDK integration and driver development standpoint, it is important to note:
Figure 2: The Zyla is available in both 4.2 Megapixel ‘Zyla 4.2’ and 5.5 Megapixel ‘Zyla 5.5’ sensor formats. Zyla 5.5 and Zyla 4.2 are available with either USB3 or 10-tap interface. The ‘Neo 5.5’ features a 5.5 Megapixel sensor, 3-tap interface and 4GB on-head memory buffer. Neo 5.5, Zyla 5.5 and Zyla 4.2 all produce data rates of up to ~ 850 MB/s at frame rates of 100 fps.
Maximum sustained frame rates relate to how fast data can be retrieved from the camera.
Additional processing overheads, including saving data to the hard drive, could impact these figures.
4.2 – Neo 5.5 and Zyla 5.5 USB3 kinetic series tests
Table 2 outlines the PC solutions for Neo 5.5 and Zyla 5.5 USB3 sCMOS that were in-house tested by Andor over extensive kinetic series lengths. The test utilised Solis acquisition software, internally triggered, rolling shutter and global shutter mode and 200 MHz/560MHz pixel readout speed, in both 12-bit (single amplifier) and 16-bit (dual amplifier) data range configurations.
Figure 3: Dell T7600 PC (System A) 4 x 250 GB Solid State Drives (SSD) configured in RAID 0 are utilised for data spooling. An additional fifth hard drive is assigned to the operating system (Windows 7, 64-bit).
The PC configuration is based on the Dell T5810 with 3.5 GHz Quad Core. This system utilizes 64 GB RAM (8 x 8 GB) for direct data spooling. Solis acquisition software is a 64-bit application and thus allows the user to utilise the full RAM capacity. It is important to note that spooling to RAM does not allow for permanent storgae and the user would need to send the data to a hard drive with equivalent capacity following data acquisition in order to save the kinetic series.
Table 2: PC solution for Neo 5.5 and Zyla USB3 sCMOS. This PC configuration with 64 GB RAM can acquire 6000 full frame (2560 x 2160 for Neo 5.5 and Zyla 5.5 and 2048 x 2048 for Zyla 4.2) 12-bit images before the full capacity is utilised. For smaller ROI’s this limit is significantly reduced.
Dell T5810 (RAM System)
Intel Xeon CPU E5-1620 v3 @ 3.5GHz quad core
System A makes use of the RAID 0 SSD configuration for direct spooling.
System B utilizes 64 GB of PC RAM for direct spooling.
Solis acquisition software is a 64-bit application and thus allows the user to utilise the full RAM capacity.
It is important to note that spooling to RAM does not allow for permanent storage and the user would need to send the data to a hard drive with equivalent capacity following data acquisition in order to save the kinetic series.
In order to drive fastest data spooling rates, especially from 10- tap Zyla sCMOS, it is highly recommended that acquisition PCs are dedicated for this purpose and that other functions or applications are not being performed in the background, particularly during data acquisition. Tests represented in this technical note were under such stringent conditions. Both SDK 3 and Solis are compatible with a 64-bit OS. A 64-bit OS is recommended for most efficient spooling, where possible.
Note however that iQ software requires a 32-bit OS
64-bit 3rd party software packages must use the 64-bit SDK3 and operate on 64-bit OS
64-bit software can spool directly to PC RAM that is greater than 4 GB
RAMDISK is the ONLY way to make use of > 4 GB PC RAM in a 32-bit application