Postprocessing Tools

Inserting instrument characteristics

This file provides a class to insert instrument specific features to a simulated image. The VIS instrument model is taken from support.VISinstrumentModel.VISinformation function.

The class supports multiprocessing.

Note

The output images will be compressed with gzip to save disk space.

Warning

The logging module used does not work well with multiprocessing, but starts to write multiple entries after a while. This should be fixed.

requires:PyFITS requires:NumPy requires:CDM03 (FORTRAN code, f2py -c -m cdm03 cdm03.f90) author:Sami-Matias Niemi contact:smn2@mssl.ucl.ac.uk version:0.9

class postproc.postprocessing.PostProcessing(values, work_queue, result_queue, seed)

Euclid Visible Instrument postprocessing class. This class allows to add radiation damage (as defined by the CDM03 model) and add readout noise to a simulated image.

applyLinearCorrection(image)

Applies a linear correction after one forward readout through the CDM03 model.

Bristow & Alexov (2003) algorithm further developed for HST data processing by Massey, Rhodes et al.

Parameters:image (ndarray) – radiation damaged image Returns:corrected image after single forward readout Return type:ndarray

applyRadiationDamage(data, iquadrant=0)

Apply radian damage based on FORTRAN CDM03 model. The method assumes that input data covers only a single quadrant defined by the iquadrant integer.

Parameters:

• data (ndarray) – imaging data to which the CDM03 model will be applied to.
• iquandrant (int) – number of the quadrant to process

cdm03 - Function signature:

sout = cdm03(sinp,iflip,jflip,dob,rdose,in_nt,in_sigma,in_tr,[xdim,ydim,zdim])

Required arguments:

sinp : input rank-2 array(‘f’) with bounds (xdim,ydim) iflip : input int jflip : input int dob : input float rdose : input float in_nt : input rank-1 array(‘d’) with bounds (zdim) in_sigma : input rank-1 array(‘d’) with bounds (zdim) in_tr : input rank-1 array(‘d’) with bounds (zdim)

Optional arguments:

xdim := shape(sinp,0) input int ydim := shape(sinp,1) input int zdim := len(in_nt) input int

Return objects:

sout : rank-2 array(‘f’) with bounds (xdim,ydim)

Note

Because Python/NumPy arrays are different row/column based, one needs to be extra careful here. NumPy.asfortranarray will be called to get an array laid out in Fortran order in memory. Before returning the array will be laid out in memory in C-style (row-major order).

Returns:image that has been run through the CDM03 model Return type:ndarray

applyReadoutNoise(data)

Applies readout noise. The noise is drawn from a Normal (Gaussian) distribution. Mean = 0.0, and std = readout.

Parameters:data (ndarray) – input data to which the readout noise will be added to Returns:updated data, noise image Return type:dict

compressAndRemoveFile(filename)

This method compresses the given file using gzip and removes the parent from the file system.

Parameters:filename (str) – name of the file to be compressed Returns:None

cutoutRegion(data)

Cuts out a region from the imaging data. The cutout region is specified by xstart/stop and ystart/stop that are read out from the self.values dictionary. Also checks if there are values that are above the given cutoff value and sets those pixels to a max value (default=33e3).

Parameters:

• data (ndarray) – image array
• max (int or float) – maximum allowed value [default = 33e3]

Returns:

cut out image from the original data

Return type:

ndarray

discretisetoADUs(data)

Convert floating point arrays to integer arrays and convert to ADUs. Adds bias level after converting to ADUs.

Parameters:data (ndarray) – data to be discretised to. Returns:discretised array in ADUs Return type:ndarray

generateCTImap(CTIed, originalData)

Calculates a map showing the CTI effect. This map is being generated by dividing radiation damaged image with the original data.

Parameters:

• CTIed (ndarray) – Radiation damaged image
• originalData (ndarray) – Original image before any radiation damage

Returns:

CTI map (ratio of radiation damaged image and original data)

Return type:

ndarray

loadFITS(filename, ext=0)

Loads data from a given FITS file and extension.

Parameters:

• filename (str) – name of the FITS file
• ext (int) – FITS header extension [default=0]

Returns:

data, FITS header, xsize, ysize

Return type:

dict

radiateFullCCD(fullCCD, quads=(0, 1, 2, 3), xsize=2048, ysize=2066)

This routine allows the whole CCD to be run through a radiation damage mode. The routine takes into account the fact that the amplifiers are in the corners of the CCD. The routine assumes that the CCD is using four amplifiers.

Parameters:

• fullCCD (ndarray) – image of containing the whole CCD
• quads (list) – quadrants, numbered from lower left

Returns:

radiation damaged image

Return type:

ndarray

run()

This is the method that will be called when multiprocessing.

writeFITSfile(data, output, unsigned16bit=True)

Write out FITS files using PyFITS.

Parameters:

• data (ndarray) – data to write to a FITS file
• output (string) – name of the output file
• unsigned16bit (bool) – whether to scale the data using bzero=32768

Returns:

None

Generating a CCD mosaic

This file contains a class to create a single VIS CCD image from separate files one for each quadrant.

requires:NumPy requires:PyFITS author:Sami-Matias Niemi contact:s.niemi@ucl.ac.uk

To execute:

python tileCCD.py -f 'Q*science.fits' -e 1

where -f argument defines the input files to be tiled and the -e argument marks the FITS extension from which the imaging data are being read.

version:0.5

Todo

1. Does not deal properly with multiple WCSs coming in the different quadrants (should recalculate the centre of the CCD and modify the WCS accordingly).
2. Improve the history section.

class postproc.tileCCD.tileCCD(inputs, log)

Class to create a single VIS CCD image from separate quadrants files.

readData()

Reads in data from all the input files and the header from the first file. Input files are taken from the input dictionary given when class was initiated.

Subtracts the pre- and overscan regions if these were simulated. Takes into account which quadrant is being processed so that the extra regions are subtracted correctly.

runAll()

Wrapper to perform all class methods.

tileCCD(xsize=2048, ysize=2066)

Tiles quadrants to form a single CCD image.

Assume that the input file naming convention is Qx_CCDX_CCDY_name.fits.

Parameters:

• xsize (int) – length of a quadrant in column direction
• ysize (int) – length of a quadrant in row direction

Returns:

image array of size (ysize*2, xsize*2)

Return type:

ndnarray

writeFITSfile(data=None, unsigned16bit=True)

Write out FITS files using PyFITS.

Parameters:

• data (ndarray) – data to write to a FITS file, if None use self.data
• unsigned16bit (bool) – whether to scale the data using bzero=32768

Returns:

None

Generating an FPA mosaic

This file contains a class to create a single VIS FPA image from separate files one for each CCD.

requires:NumPy requires:PyFITS author:Sami-Matias Niemi contact:s.niemi@ucl.ac.uk

To execute:

python tileFPA.py -f 'CCD*science.fits' -e 1

where -f argument defines the input files to be tiled and the -e argument marks the FITS extension from which the imaging data are being read.

version:0.1

class postproc.tileFPA.tileFPA(inputs, log)

Class to create a single VIS FPA image from separate CCD files.

readData()

Reads in data from all the input files and the header from the first file. Input files are taken from the input dictionary given when class was initiated.

Subtracts the pre- and overscan regions if these were simulated. Takes into account which quadrant is being processed so that the extra regions are subtracted correctly.

runAll()

Wrapper to perform all class methods.

tileFPA(xgap=1.643, ygap=8.116)

Tiles quadrants to form a single CCD image.

Assume that the input file naming convention is Qx_CCDX_CCDY_name.fits.

Parameters:

• xgap (float) – length of the gap between in mm two CCDs in column direction [default=1.643]
• ygap (float) – length of the gap between in mm two CCDs in row direction [default=8.116]

Returns:

FPA image array

Return type:

ndnarray

writeFITSfile(data=None, unsigned16bit=True, ra=299.8680417, dec=40.73388889)

Write out FITS files using PyFITS.

Parameters:

• data (ndarray) – data to write to a FITS file, if None use self.data
• unsigned16bit (bool) – whether to scale the data using bzero=32768
• ra (float) – Right Ascension of the centre of the FPA [default = Cygnus A]
• dec (float) – Declination of the centre of the FPA [default = Cygnus A]

Returns:

None