Data Model: The ROSA/HARDCAM Rapid Imagers

Instrument Description

The ROSA and HARDCAM instruments are the rapid imaging arrays at the Dunn Solar Telescope. The instrument is configurable to cover several wavelengths of interest with a variety of different cameras. The typical setup is described below:

Camera	Typical Wavelength	FWHM	Chip Size	Typical Plate Scale	Typical Framerate
Andor Zyla	Hydrogen Alpha 6563Å	0.25Å	2048x2048	0.0845"/pix	24 Hz
Andor iXon "das1"	4300Å (G-Band)	Broadband	1002x1004	0.06 | 0.16"/pix	30 Hz
Andor iXon "das2"	4170Å (Continuum)	Broadband	1002x1004	0.06 | 0.16"/pix	30 Hz
Andor iXon "das3"	Calcium K 3934Å	1.2Å	1002x1004	0.06 | 0.16"/pix	12, 15, 30 Hz
Andor iXon "das4"	3500Å Continuum, H-beta	Broadband	1002x1004	0.06 | 0.16"/pix	6 Hz
Andor iXon "CSUN"*	PI	512x512	PI
Andor "HARDCAM v1"*	PI	512x512	PI

* These cameras have largely been superseded by the das3 and Zyla arrays respectively. At PI request, they can be used in conjunction with the other systems.

These are the most common data modes as of January 2024. Deviations from this setup will be noted in the fits headers and data structures. The das1-4 cameras are typically operated in diffraction limited mode or a wider FOV mode. Prior to July 2023, the wider-FOV mode was only used in Parker Solar Probe Footpoint campaigns. After July 2023, the wider FOV began to be used for active region studies, in order to better capture flaring regions.

Data File Description & Summary

Level-0 General Description

Zyla Camera

The new Andor Zyla camera is typically run at the core of the H-alpha 6563Å line. Level-0 data files are contained in type-tagged directories of the form:

DBJ_data_[TIME]_[TARGET/TYPE]

Or similar. These times are estimates, and the target type can vary. The start time should not be determined from the folder name. Currently (January 2024), the Zyla camera is set to trigger from the ROSA cameras. The better way to determine the start time of an observing series is to find the corresponing ROSA series, and determine the precise time from the ROSA cameras. This is accurate to within 1 second.

Within each directory, files are stored as platform-native binary files with the .dat data type. In some older datasets where the files have been compressed, they may be in compressed FITS format, .fits.fz, readable natively with astropy and most FITS handlers. Files are numbered leading by the least significant digit. So the numbering is in the form:

0000000000.dat, 1000000000.dat, 2000000000.dat, ... 9000000000.dat, 0100000000.dat

Fits files are named in the same manner, with the addition of the .fits.fz tag.

Zyla binary files can be read with the routines included in SSOsoft from binary, or manually. Manual reading requires the size of the chip be known, along with the data type. The datatype for the Zyla chip is an unsigned 16-bit integer. Unless otherwise note, for the full chip, the dimensions are 2050x2060, with overscan rows that can be eliminated to form a 2048x2048 image. Deviations from this will be noted in the scanned observing logs contained in the observing day's root directory. SSOsoft routines perform overscan windowing automatically upon data read. An image can be formed manually in python 3+ by, e.g.:

import numpy as np
image = np.fromfile("DBJ_data_143000_observation/0000000000.dat", dtype=np.uint16)
image = image.reshape(2050, 2060)

Zyla compressed fits files have only minimal header information, with data contained in the second entry in the FITS HDU List. In python, this is extension 1.

ROSA das1-4

The ROSA cameras are centrally synced. All will have the same start times, and for slower channels, such as Ca K, the master sync still handles the cadence. So for a 15 and 30 fps camera, timestamps will still align on every other frame of the higher cadence channel. The ROSA camera system writes files in FITS format with the .fit file extension, which may be compressed via fpack to the .fit.fz file extension. Dark and flat files are tagged in the file name, but other calibrations, such as the line grids, target images, etc., are not tagged in the file names. Typically, observing series will be comprised of many files, while calibration series will contain only a few files.

Each file has one empty primary extension containing a master header, and up to 256 data extensions.

HDU Name	HDU Index	Units	Description
Primary	0	N/A	Header only, contains camera information
Image	1 -- 256	DN	Image Extensions

PRIMARY HEADER DESCRIPTION

Key	Value (Ex.)	Type	Comment
HEIRARCH CAMERA SERIAL	2996	INT	Camera serial number
EXPOSURE	0.0100 Sec	STR	Exposure time in seconds.
HEIRARCH FRAME RATE	33 mSec	STR	The camera cadence in milliseconds. The frame rate is 1/this number
HEIRARCH BASELINE CLAMP	On	STR	Not entirely sure, but I think this indicates that the sync unit is connected
HEIRARCH CAMERA GAIN	0	INT
HEIRARCH CAMERA TEMPERATURE	-73C	STR	Camera temperature in C. If the Peltier coolers are functioning, this should be below -40C
HEIRARCH OBSERVATION TYPE	NORMAL, FLAT, or DARK	STR
HEIRARCH BINNING X/Y	1	INT	Binning is typically 1
HEIRARCH START X/Y	1	INT	For windowing the chip. Rarely used
HEIRARCH FINISH X/Y	1004/1002	INT	For binning the chip. Depends on X/Y axis
SHUTTER	OPEN	STR	Should be open for all except dark frames

IMAGE EXTENSION HEADER DESCRIPTION

Key	Value (Ex.)	Type	Comment
HEIRARCH FRAME NUMBER	1	INT	Index of image frame since start of observing series
DATE	2023-12-06T15:50:06.433	DATETIME	Note that the DATE keywork is typically used to signify the file creation date. ROSA headers are not in compliance in this way. Also, cameras das3 and 4 are affected by daylight savings time
HEIRARCH FRAME DELTA	0.033122 Secs	TIMEDELTA	Amount of time passed since the previous frame was captured

Level-1 and Level-1.5 Description

The level-1 data pipeline is handled through the original SSOsoft package, and provides dark/flat correction, and speckle-reconstruction via the KISIP v0.6 code. The Level-1.5 pipeline is handled through a fork of the SSOsoft package, and contains a python-based destretch algorithm.

As a summary: speckle reconstruction removes residual atmospheric affects not corrected for by adaptive optics. It does this via a triple-correlation. It's essentially a Fourier deconvolution. The underlying assumption is that the individual images are of a short enough exposure that the atmospheric distortions are "froze", not changing over the course of the exposure, and that a stack of sequential images contains very minor evolution that can be characterized and detrended. The KISIP code works on image subfields for parallelization, splitting a data cube into smaller sections and detrending the atmosphere on each before recombining the subfields. Due to the nature of the correction, this can result in shifts of structure within these subfields image-to-image in addition to those brought on be atmospheric distortion.

Destretch was developed as a low-level correction for atmospheric distortion, but finding instances of structures moving, and warping the image to remove those shifts. When applied to speckled data, it corrects for atmospheric warp as well as that induced by the speckle reconstruction.

The SSOsoft package was updated in 2023 to include the destretch corrections. All reduced data taken in 2022 or later has had some version of the destretch code applied, and select datasets from earlier have been destretched as well. The SSOsoft destretch module can be applied to older datasets, either by end-users, or by the current support astronomer upon request.

As of 2024, the newest versions of SSOsoft include updated headers and perform translations for each camera to align them with the telescope rotation angle. Updated headers are expected to be provided to older datasets as part of ongoing efforts to standardize the archive, however, due to the flexible nature of the instrument, the translations may not be available for older datasets. Those that require no extra manipulation have the header keyword pair "PRSTEP3 'ALIGN TO SOLAR NORTH / SSOsoft" present.

All ROSA/HARDcam Level-1 and 1.5 data are packaged in FITS format with a single image extension per file.

CURRENT HEADER DESCRIPTION

Headers from the most current releases of SSOsoft will have the following information:

Key	Example Value	Comment
HEIRARCH FRAME NUMBER	64	ROSA ONLY, last frame number of the burst used for reconstuction
STARTOBS, DATE_OBS, and DATE-BEG	YYYY-MM-DDTHH:MM:SS.FFF	All three keywords are used thanks to our pal Competing Standards. Older versions of SSOsoft may use the DATE keyword for the observation start instead.
DATE	YYYY-MM-DDTHH:MM:SS	Date of file creation. In older versions, this was used at the observing date.
HEIRARCH FRAME DELTA	0.033046 Sec	ROSA ONLY, time between frames of the speckled observations
CRVAL1	123.456	WCS standard, Solar-X value in the HPLN-TAN frame
CRVAL2	789.101	WCS standard, Solar-Y value in the HPLT_TAN frame
CTYPE1	HPLN_TAN
CTYPE2	HPLT_TAN
CUNIT1	arcsec	DST reports coordinates as Stonyhurst Lat/Lon by default. SSOsoft transforms to Solar-X/Y for ease of use
CUNIT2	arcsec
CDELT1	0.156	Determined from line grid images.
CDELT2	0.156
CRPIX1	501.0 or 1024.0	Depending on the camera. Should be the middle of the frame
CRPIX2	502.0 or 1024.0	Note that these coordinates will still be offset from other observatories. The nature of the DST makes coordinate determination a somewhat hazy affair.
CROTAN	12.345	Rotation from solar north in degrees. Typically, this is kept near 0
SCINT	1.234	Average reading from the telescope's Seykora scintillation monitor during image aquisition. Does not account for AO or speckle.
LLVL	1.234	Unitless quantity. Light level as seen from the tower. For precise photometry, the curve should be detrended from image sequence.
RSUN_ARC	1927.07	What DST thinks the diameter of the sun is in arcseconds
NSUMEXP	64	Number of frames used in speckle reconstruction
TEXPOSUR	10	Milliseconds, single-frame exposure time
AUTHOR	sellers	Person who wrote the fits file, so you can hunt them for sport.
TELESCOP	DST	DST
INSTR	ROSA or HARDCAM
WAVE	678	Wavelength of filter
WAVEUNIT	nm	Unit of wavelength
SPKLALPH	0.0123	Alpha parameter, related to atmospheric Fried parameter, estimated by KISIP. I... Haven't worked out the units on this yet, or it would be a more useful quantity.
PRSTEP1, 2, 3, 4, 5	CORRECTION DONE	Names of all the corrections performed on the file

While standardization efforts are underway, they are by no means complete. Previous iterations of Level-1 and 1.5 files with the "sellers" author tag may lack pointing, scintillation, light level, wavelength, and a handful of other keywords. Files with "ONDREJP" as the author should contain, at minimum, timing information.

If your dataset has an older header version, and you would like it to be prioritized for updates, please contact the support astronomer to request the dataset be moved up in the queue.

Example of data at different levels

Below are examples of the ROSA G-Band and HARDCam H-alpha Level-0, Level-1, and Level-1.5 data products. These snapshots were randomly selected from data taken on 2023-10-19, and corresponds to times of 15:03:23 (UTC, G-Band) and 15:01:12 (UTC, H-alpha). The Level-1.5 headers are displayed below.

FITS HEADERS FROM Level-1.5 FILES