Data reduction tools

VIS Data Reduction and Processing

This simple script can be used to reduce (simulated) VIS data.

The script was initially written for reducing a single CCD data. However, since the version 0.5 the script tries to guess if the input is a single quadrant then reduce correctly.

The script performs the following data reduction steps:

1 Bias correction
2 CTI correction
3 Flat fielding (only if an input file is provided)

To Run:

python reduceVISdata.py -i VISCCD.fits -b superBiasVIS.fits -f SuperFlatField.fits

requires:PyFITS requires:NumPy requires:CDM03 (FORTRAN code, f2py -c -m cdm03 cdm03.f90) author:Sami-Matias Niemi contact:smn2@mssl.ucl.ac.uk version:0.5

Todo

1. implement background/sky subtraction

class reduction.reduceVISdata.reduceVISdata(values, log)

Simple class to reduce VIS data.

applyCTICorrection()

Applies a CTI correction in electrons using CDM03 CTI model. Converts the data to electrons using the gain value given in self.values. The number of forward reads is defined by self.values[‘order’] parameter.

Bristow & Alexov (2003) algorithm further developed for HST data processing by Massey, Rhodes et al.

There is probably an excess of .copy() calls here, but I had some problems when calling the Fortran code so I added them for now.

flatfield()

Take into account pixel-to-pixel non-uniformity through multiplicative flat fielding.

subtractBias()

Simply subtracts self.bias from the input data.

writeFITSfile()

Write out FITS files using PyFITS.

VIS data reduction studies

Bias Calibration

This simple script can be used to study the number of bias frames required to meet the VIS calibration requirements.

The following requirements related to the bias calibration has been taken from GDPRD.

R-GDP-CAL-052: The contribution of the residuals of VIS bias subtraction to the error on the determination of each ellipticity component of the local PSF shall not exceed 3x10-5 (one sigma).

R-GDP-CAL-062: The contribution of the residuals of VIS bias subtraction to the relative error sigma(R2)/R2 on the determination of the local PSF R2 shall not exceed 1x10-4 (one sigma).

requires:PyFITS requires:NumPy requires:matplotlib requires:VISsim-Python version:0.95 author:Sami-Matias Niemi contact:smn2@mssl.ucl.ac.uk

analysis.biasCalibration.addReadoutNoise(data, readnoise=4.5, gain=3.1, number=1)

Add readout noise to the input data. The readout noise is the median of the number of frames.

Parameters:

• data (ndarray) – input data to which the readout noise will be added to [ADUs]
• readnoise (float) – standard deviation of the read out noise [electrons]
• gain (float) – the gain factor that is used to convert electrons to ADUs
• number (int) – number of read outs to median combine before adding to the data [default=1]

Returns:

data + read out noise

Return type:

ndarray [same as input data]

analysis.biasCalibration.findTolerableErrorPiston(log, file='data/psf12x.fits', oversample=12.0, samples=12, psfs=4000, sigma=0.36, iterations=5, debug=False)

Calculate ellipticity and size for PSFs of different scaling when there is a residual bias offset.

Parameters:sigma (float) – 1sigma radius of the Gaussian weighting function for shape measurements

analysis.biasCalibration.findTolerableErrorSlope(log, file='data/psf12x.fits', oversample=12.0, samples=12, psfs=4000, sigma=0.36, iterations=5, pixels=60)

Calculate ellipticity and size for PSFs of different scaling when there is a residual bias slope.

Parameters:sigma (float) – 1sigma radius of the Gaussian weighting function for shape measurements

analysis.biasCalibration.pistonKnowledge(log, file='data/psf2x.fits', oversample=2.0, psfs=1000, sigma=0.36, iterations=4, debug=False)

analysis.biasCalibration.plotDeltaEs(deltae1, deltae2, deltae, output, title='', ymax=8, req=3)

Generates a simple plot showing the errors in the ellipticity components.

analysis.biasCalibration.plotEs(deltae1, deltae2, deltae, output, title='')

Generates a simple plot showing the ellipticity components.

analysis.biasCalibration.plotNumberOfFramesDelta(results, timeStamp=False)

Creates a simple plot to combine and show the results for errors (delta).

Parameters:

• results (dict) – results to be plotted
• timeStamp (bool) – whether to include a time stamp in the output image

analysis.biasCalibration.plotNumberOfFramesSigma(results, reqe=3e-05, reqr2=0.0001, shift=0.1, timeStamp=False)

Creates a simple plot to combine and show the results.

Parameters:

• results (dict) – results to be plotted
• req (float) – the requirement
• ymax (int or float) – maximum value to show on the y-axis
• shift (float) – the amount to shift the e2 results on the abscissa (for clarity)
• timeStamp (bool) – whether to include a time stamp in the output image

analysis.biasCalibration.simpleAnalytical(offset=1500, size=(50, 50), readnoise=4.5, gain=3.1, req=0.6)

A simple function to test the area of pixels needed (defined by size) to derive the pixel offset to the level of required number of electrons given the readout noise and the gain of the system.

Parameters:

• offset (int) – the offset level in electrons [default = 1500]
• size (tuple) – area describing the number of pixels available [default = (50, 50)]
• readnoise (float) – readout noise of the full detection chain [default = 4.5]
• gain (float) – gain of the detection system [default = 3.1]
• req (float) – required level to reach in electrons [default = 0.1]

Returns:

none

analysis.biasCalibration.testBiasCalibrationDelta(log, numdata=2066, floor=995, xsize=2048, ysize=2066, order=3, biases=15, surfaces=100, file='psf1x.fits', psfs=500, psfscale=1000.0, debug=False, plots=False)

Derive the PSF ellipticities for a given number of random surfaces with random PSF positions and a given number of biases median combined and compare to the nominal PSF ellipticity.

This function can be used to derive the error (delta) in determining ellipticity and size given a reference PSF.

Choices that need to be made and effect the results:

1. bias surface that is assumed (amplitude, complexity, etc.)
2. whether the order of the polynomial surface to be fitted is known or not
3. size of the Gaussian weighting function when calculating the ellipticity components

There are also other choices such as the number of PSFs and scaling and the random numbers generated for the surface that also affect the results, however, to a lesser degree.

Generates a set of plots that can be used to inspect the simulation.

analysis.biasCalibration.testBiasCalibrationSigma(log, numdata=2066, floor=3500, xsize=2048, ysize=2066, order=3, biases=15, surfaces=100, file='psf1x.fits', psfs=500, psfscale=100000.0, gain=3.1, debug=False, plots=True)

Derive the PSF ellipticities for a given number of random surfaces with random PSF positions and a given number of biases median combined.

This function is to derive the the actual values so that the knowledge (variance) can be studied.

Choices that need to be made and effect the results:

1. bias surface that is assumed (amplitude, complexity, etc.)
2. whether the order of the polynomial surface to be fitted is known or not
3. size of the Gaussian weighting function when calculating the ellipticity components

There are also other choices such as the number of PSFs and scaling and the random numbers generated for the surface that also affect the results, however, to a lesser degree.

Generates a set of plots that can be used to inspect the simulation.

Flat Field Calibration

This simple script can be used to study the number of flat fields required to meet the VIS calibration requirements.

The following requirements related to the flat field calibration has been taken from GDPRD.

R-GDP-CAL-054: The contribution of the residuals of VIS flat-field correction to the error on the determination of each ellipticity component of the local PSF shall not exceed 3x10-5 (one sigma).

R-GDP-CAL-064: The contribution of the residuals of VIS flat-field correction to the relative error on the determination of the local PSF R2 shall not exceed 1x10-4 (one sigma).

Note

The amount of cosmic rays in the simulated input images might be too low, because the exposure was set to 10 seconds and cosmic rays were calculated based on this. However, in reality the readout takes about 80 seconds. Thus, the last row is effected by cosmic a lot more than by assuming a single 10 second exposure.

requires:PyFITS requires:NumPy requires:SciPy requires:matplotlib requires:VISsim-Python version:0.97 author:Sami-Matias Niemi contact:s.niemi@ucl.ac.uk

analysis.FlatfieldCalibration.findTolerableError(log, file='data/psf4x.fits', oversample=4.0, psfs=10000, iterations=7, sigma=0.75)

Calculate ellipticity and size for PSFs of different scaling when there is a residual pixel-to-pixel variations.

analysis.FlatfieldCalibration.generateResidualFlatField(files='Q0*flatfield*.fits', combine=77, lampfile='data/VIScalibrationUnitflux.fits', reference='data/VISFlatField1percent.fits', gain=3.5, plots=False, debug=False)

Generate a median combined flat field residual from given input files.

Randomly draws a given number (kw combine) of files from the file list identified using the files kw. Median combine all files before the lamp profile given by lampfile kw is being divided out. This will produce a derived flat field. This flat can be compared against the reference that was used to produce the initial data to derive a residual flat that describes the error in the flat field that was derived.

Parameters:

• files (str) – wildcard flagged name identifier for the FITS files to be used for generating a flat
• combine (int) – number of files to median combine
• lampfile (str) – name of the calibration unit flux profile FITS file
• reference (str) – name of the reference pixel-to-pixel flat field FITS file
• gain (float) – gain factor [e/ADU]
• plots (boolean) – whether or not to generate plots
• debug (boolean) – whether or not to produce output FITS files

Warning

Remember to use an appropriate lamp and reference files so that the error in the derived flat field can be correctly calculated.

Returns:residual flat field (difference between the generated flat and the reference) Return type:ndarray

analysis.FlatfieldCalibration.plotNumberOfFrames(results, reqe=3e-05, reqr2=0.0001, shift=0.1, outdir='results', timeStamp=False)

Creates a simple plot to combine and show the results.

Parameters:

• res (list) – results to be plotted [results dictionary, reference values]
• reqe (float) – the requirement for ellipticity [default=3e-5]
• reqr2 (float) – the requirement for size R2 [default=1e-4]
• shift (float) – the amount to shift the e2 results on the abscissa (for clarity)
• outdir (str) – output directory to which the plots will be saved to
• timeStamp (bool) – whether or not to include a time stamp to the output image

Returns:

None

analysis.FlatfieldCalibration.testFlatCalibration(log, flats, surfaces=10, file='data/psf1x.fits', psfs=5000, sigma=0.75, iterations=7, weighting=True, plot=False, debug=False)

Derive the PSF ellipticities for a given number of random surfaces with random PSF positions and a given number of flat fields median combined.

This function is to derive the the actual values so that the knowledge (variance) can be studied.

analysis.FlatfieldCalibration.testNoFlatfieldingEffects(log, file='data/psf1x.fits', oversample=1.0, psfs=500)

Calculate ellipticity and size variance and error in case of no pixel-to-pixel flat field correction.

Non-linearity I: Detection Chain

This simple script can be used to study the error in the non-linearity correction that can be tolerated given the requirements.

The following requirements related to the non-linearity have been taken from GDPRD.

R-GDP-CAL-058: The contribution of the residuals of the non-linearity correction on the error on the determination of each ellipticity component of the local PSF shall not exceed 3x10**-5 (one sigma).

R-GDP-CAL-068: The contribution of the residuals of the non-linearity correction on the error on the relative error sigma(R**2)/R**2 on the determination of the local PSF R**2 shall not exceed 1x10**-4 (one sigma).

requires:PyFITS requires:NumPy requires:SciPy requires:matplotlib requires:VISsim-Python version:0.97 author:Sami-Matias Niemi contact:s.niemi@ucl.ac.uk

analysis.nonlinearityCalibration.plotResults(results, reqe=3e-05, reqr2=0.0001, outdir='results', timeStamp=False)

Creates a simple plot to combine and show the results.

Parameters:

• res (list) – results to be plotted [reference values, results dictionary]
• reqe (float) – the requirement for ellipticity [default=3e-5]
• reqr2 (float) – the requirement for size R2 [default=1e-4]
• outdir (str) – output directory to which the plots will be saved to

Returns:

None

analysis.nonlinearityCalibration.testNonlinearity(log, file='data/psf12x.fits', oversample=12.0, sigma=0.75, phs=0.98, phases=None, psfs=5000, amps=12, multiplier=1.5, minerror=-5.0, maxerror=-1, linspace=False)

Function to study the error in the non-linearity correction on the knowledge of the PSF ellipticity and size.

The error has been assumed to follow a sinusoidal curve with random phase and a given number of angular frequencies (defined by the multiplier). The amplitudes being studied, i.e. the size of the maximum deviation, can be spaced either linearly or logarithmically.

Parameters:

• log (instance) – logger instance
• file (str) – name of the PSF FITS files to use [default=data/psf12x.fits]
• oversample – the PSF oversampling factor, which needs to match the input file [default=12]
• sigma (float) – 1sigma radius of the Gaussian weighting function for shape measurements
• phs (float) – phase in case phases = None
• phases (None or int) – if None then a single fixed phase will be applied, if an int then a given number of random phases will be used
• psfs (int) – the number of PSFs to use.
• amps (int) – the number of individual samplings used when covering the error space
• multiplier (int or float) – the number of angular frequencies to be used
• minerror (float) – the minimum error to be covered, given in log10(min_error) [default=-5 i.e. 0.001%]
• maxerror (float) – the maximum error to be covered, given in log10(max_error) [default=-1 i.e. 10%]
• linspace (boolean) – whether the amplitudes of the error curves should be linearly or logarithmically spaced.

Returns:

reference value and results dictionaries

Return type:

list

Testing the CTI Correction Algorithm

This script can be used to test the CTI correction algorithm performance.

requires:NumPy requires:PyFITS requires:matplotlib version:0.3 author:Sami-Matias Niemi contact:s.niemi@ucl.ac.uk

analysis.testCTIcorrection.addCTI(file, loc=1900, bcgr=72.2, thibautCDM03=False, beta=True, serial=1, parallel=1)

Add CTI trails to a FITS file or input data.

analysis.testCTIcorrection.cutoutRegions(files, xcen=1900, ycen=1900, side=140)

Parameters:

• files –
• xcen –
• ycen –
• side –

Returns:

analysis.testCTIcorrection.plotResults(results)

Plot the CTI correction algorithm results.

Parameters:results – CTI test results Returns:None

analysis.testCTIcorrection.plotResultsNoNoise(inputfile, title, bins=10)

Plot the CTI correction algorithm results.

Returns:None

analysis.testCTIcorrection.testCTIcorrection(log, files, sigma=0.75, iterations=4, xcen=1900, ycen=1900, side=20)

Calculates PSF properties such as ellipticity and size from data without CTI and from CTI data.

Parameters:

• log (instance) – python logger instance
• files (list) – a list of files to be processed
• sigma (float) – size of the Gaussian weighting function
• iterations (int) – the number of iterations for the moment based shape estimator
• xcen (int) – x-coordinate of the object centre
• ycen (int) – y-coordinate of the object centre
• side (int) – size of the cutout around the centre (+/- side)

Returns:

ellipticity and size

Return type:

dict

analysis.testCTIcorrection.testCTIcorrectionNonoise(log, files, output, sigma=0.75, iterations=4)

Calculates PSF properties such as ellipticity and size from data w/ and w/o CTI.

Parameters:

• log (instance) – python logger instance
• files (list) – a list of files to be processed
• sigma (float) – size of the Gaussian weighting function
• iterations (int) – the number of iterations for the moment based shape estimator

Returns:

ellipticity and size

Return type:

dict

analysis.testCTIcorrection.useThibautsData(log, output, bcgr=72.2, sigma=0.75, iterations=4, loc=1900, galaxies=1000, datadir='/Users/smn2/EUCLID/CTItesting/uniform/', thibautCDM03=False, beta=False, serial=1, parallel=1)

Test the impact of CTI in case of no noise and no correction.

Parameters:

• log – logger instance
• bcgr – background in electrons for the CTI modelling
• sigma – size of the weighting function for the quadrupole moment
• iterations – number of iterations in the quadrupole moments estimation
• loc – location to which the galaxy will be placed [default=1900]
• galaxies – number of galaxies to use (< 10000)
• datadir – directory pointing to the galaxy images

Returns:

Cosmic Ray Rejection

This simple script can be used to study the error in the cosmic ray rejection that can be tolerated given the requirements.

The following requirements related to the cosmic rays have been taken from CalCD-B.

Note that the analysis is for a single star. Thus, if we consider a single exposure we can relax the requirements given that there will be on average about 1850 stars in each field that are usable for PSF modelling. Furthermore it shuold be noted that the presence of cosmic rays will always increase the size. Because of this one cannot combine the contribution from cosmic rays with other effects by adding each individual contribution in quadrature. It is more appropriate to add the impact of cosmic rays to the size of the PSF linearly given the preferred direction.

requires:PyFITS requires:NumPy requires:SciPy requires:matplotlib requires:VISsim-Python version:0.4 author:Sami-Matias Niemi contact:s.niemi@ucl.ac.uk

analysis.cosmicrayCalibration.plotResults(results, reqe=3e-05, reqr2=0.0001, outdir='results', timeStamp=False)

Creates a simple plot to combine and show the results.

Parameters:

• res (list) – results to be plotted [reference values, results dictionary]
• reqe (float) – the requirement for ellipticity [default=3e-5]
• reqr2 (float) – the requirement for size R2 [default=1e-4]
• outdir (str) – output directory to which the plots will be saved to

Returns:

None

analysis.cosmicrayCalibration.testCosmicrayRejection(log, file='data/psf1x.fits', oversample=1.0, sigma=0.75, psfs=20000, scale=1000.0, min=1e-05, max=50, levels=15, covering=1.4, single=False)

This is for a single PSF.

Parameters:

• log –
• file –
• oversample –
• sigma –
• psfs –
• scale –
• min –
• max –
• levels –
• covering –
• single –

Returns:

analysis.cosmicrayCalibration.testCosmicrayRejectionMultiPSF(log, file='data/psf1x.fits', oversample=1.0, sigma=0.75, psfs=2000, scale=1000.0, min=0.0001, max=100.0, levels=9, covering=2.0, stars=1850, single=False)

Parameters:

• log –
• file –
• oversample –
• sigma –
• psfs –
• scale –
• min –
• max –
• levels –
• covering –
• single –

Returns:

Impact of Ghost Images

This scripts can be used to study the impact of ghost images on the weak lensing measurements.

requires:PyFITS requires:NumPy (1.7.0 or newer for numpy.pad) requires:SciPy requires:matplotlib requires:VISsim-Python version:0.2 author:Sami-Matias Niemi contact:s.niemi@ucl.ac.uk

analysis.analyseGhosts.analyseInFocusImpact(log, filename='data/psf4x.fits', psfscale=100000, maxdistance=100, oversample=4.0, psfs=1000, iterations=6, sigma=0.75)

Calculates PSF size and ellipticity when including another PSF scaled to a given level (requirement = 5e-5)

Parameters:

• log –
• filename – name of the PSF file to analyse
• psfscale – level to which the original PSF is scaled to
• maxdistance – maximum distance the ghost image can be from the original PSF (centre to centre)
• oversample – oversampling factor
• psfs – number of PSFs to analyse (number of ghosts in random locations)
• iterations – number of iterations in the shape measurement
• sigma – size of the Gaussian weighting function

Returns:

results

Return type:

dict

analysis.analyseGhosts.analyseOutofFocusImpact(log, filename='data/psf4x.fits', psfscale=100000, maxdistance=100, inner=8, outer=60, oversample=4.0, psfs=5000, iterations=5, sigma=0.75, lowghost=-7, highghost=-2, samples=9)

Calculates PSF size and ellipticity when including an out-of-focus doughnut of a given contrast level. The dougnut pixel values are all scaled to a given scaling value (requirement 5e-5).

Parameters:

• log – logger instance
• filename – name of the PSF file to analyse
• psfscale – level to which the original PSF is scaled to
• maxdistance – maximum distance the ghost image can be from the original PSF (centre to centre)
• inner – inner radius of the out-of-focus doughnut
• outer – outer radius of the out-of-focus doughnut
• oversample – oversampling factor
• psfs – number of PSFs to analyse (number of ghosts in random locations)
• iterations – number of iterations in the shape measurement
• sigma – size of the Gaussian weighting function
• lowghost – log of the highest ghost contrast ratio to study
• highghost – log of the lowest ghost contrast ratio to study
• samples – number of points for the contrast ratio to study

Returns:

results

Return type:

dict

analysis.analyseGhosts.deleteAndMove(dir, files='*.fits')

Parameters:

• dir –
• files –

Returns:

analysis.analyseGhosts.drawDoughnut(inner, outer, oversample=1, plot=False)

Draws a doughnut shape with a given inner and outer radius.

Parameters:

• inner (float) – inner radius in pixels
• outer (float) – outer radius in pixels
• oversample (int) – oversampling factor [default=1]
• plot (bool) – whether to generate a plot showing the doughnut or not

Returns:

image, xcentre, ycentre

Return type:

list

analysis.analyseGhosts.objectDetection(log, magnitude=24.5, exptime=565, exposures=3, fpeak=0.7, offset=0.5, covering=2550, ghostlevels=(6e-07, 1e-06, 4e-06, 5e-06, 1e-05, 5e-05))

Derive area loss in case of object detection i.e. the SNR drops below the requirement. Assumes that a ghost covers the covering number of pixels and that the peak pixel of a point source contains fpeak fraction of the total counts. An offset between the V-band and VIS band is applied as VIS = V + offset.

Parameters:

• log – logger instance
• magnitude – magnitude limit of the objects to be detected
• exptime – exposure time of an individual exposure in seconds
• exposures – number of exposures combined in the data analysis
• fpeak – PSF peak fraction
• offset – offset between the V-band and the VIS band, VIS = V + offset
• covering – covering fraction of a single ghost in pixels
• ghostlevels – a tuple of ghost levels to inspect
• ghostlevels – tuple

Returns:

None

analysis.analyseGhosts.plotGhostContribution(res, title, output, title2, output2)

analysis.analyseGhosts.plotGhostContributionElectrons(log, res, title, output, title2, output2, req=0.001)

analysis.analyseGhosts.shapeMeasurement(log)

Shape measurement bias as a result of ghosts.

Non-linearity II: Model Transfer

Most PSF stars are bright while most weak lensing galaxies are faint, thus the PSF model needs to be transferable to an appropriate object luminosity (and colour, position, etc.). This simple script can be used to study the error in the model transfer due to non-linearity that can be tolerated given the requirements.

The following requirements related to the model transfer have been taken from CalCD-B.

requires:PyFITS requires:NumPy requires:SciPy requires:matplotlib requires:VISsim-Python version:0.2 author:Sami-Matias Niemi contact:s.niemi@ucl.ac.uk

analysis.nonlinearityModelTransfer.plotResults(results, reqe=3.5e-05, reqr2=0.00035, outdir='results', timeStamp=False)

Creates a simple plot to combine and show the results.

Parameters:

• res (list) – results to be plotted [reference values, results dictionary]
• reqe (float) – the requirement for ellipticity [default=3.5e-5]
• reqr2 (float) – the requirement for size R2 [default=3.5e-4]
• outdir (str) – output directory to which the plots will be saved to

Returns:

None

analysis.nonlinearityModelTransfer.testNonlinearityModelTransfer(log, file='data/psf12x.fits', oversample=12.0, sigma=0.75, psfs=5000, amps=12, multiplier=1.5, minerror=-5.0, maxerror=-1.0, lowflux=500, highflux=180000, linspace=False)

Function to study the error in the non-linearity correction on the knowledge of the PSF ellipticity and size.

The error has been assumed to follow a sinusoidal curve with random phase and a given number of angular frequencies (defined by the multiplier). The amplitudes being studied, i.e. the size of the maximum deviation, can be spaced either linearly or logarithmically.

Parameters:

• log (instance) – logger instance
• file (str) – name of the PSF FITS files to use [default=data/psf12x.fits]
• oversample – the PSF oversampling factor, which needs to match the input file [default=12]
• sigma (float) – 1sigma radius of the Gaussian weighting function for shape measurements
• phs (float) – phase in case phases = None
• phases (None or int) – if None then a single fixed phase will be applied, if an int then a given number of random phases will be used
• psfs (int) – the number of PSFs to use.
• amps (int) – the number of individual samplings used when covering the error space
• multiplier (int or float) – the number of angular frequencies to be used
• minerror (float) – the minimum error to be covered, given in log10(min_error) [default=-5 i.e. 0.001%]
• maxerror (float) – the maximum error to be covered, given in log10(max_error) [default=-1 i.e. 10%]
• linspace (boolean) – whether the amplitudes of the error curves should be linearly or logarithmically spaced.

Returns:

reference value and results dictionaries

Return type:

list

This script can be used to derive statistics of background pixels.

analysis.analyseBackground.dilation(data)

Use dilation to define the background. Not working too well...

analysis.analyseBackground.kde_scipy(x, x_grid, bandwidth=0.2, **kwargs)

Kernel Density Estimation with Scipy

analysis.analyseBackground.kde_sklearn(x, x_grid, bandwidth=0.2, **kwargs)

Kernel Density Estimation with Scikit-learn

analysis.analyseBackground.kde_statsmodels_m(x, x_grid, bandwidth=0.2, **kwargs)

Multivariate Kernel Density Estimation with Statsmodels

analysis.analyseBackground.kde_statsmodels_u(x, x_grid, bandwidth=0.2, **kwargs)

Univariate Kernel Density Estimation with Statsmodels

analysis.analyseBackground.loadData(filename)

Load data, exclude pre- and over scan regions, and subtract the bias level found from the header.

analysis.analyseBackground.maskObjects(data, sigma=4.0, iterations=None)

Mask objects using sigma clipping around the median.

Will also plot the data and the mask.

Returns:masked numpy array

analysis.analyseBackground.plotEntropy(filename)

Plots the entropy in the data. No really suitable as the input data not in a useful format.

analysis.analyseBackground.plotStatistcs(data)

Plot statistics of the background pixels. Assumes that the input contains only the background pixel values as a 1D array.

Returns:None

analysis.analyseBackground.sigmaClippedOpening(data)

Perform Gaussian filtering, sigma clipping and binary opening to define a mask for the background pixels.

Background Subtraction

This scripts can be used to study the impact of background subtraction errors on the shape measurements.

requires:NumPy requires:SciPy requires:matplotlib requires:VISsim-Python version:0.1 author:Sami-Matias Niemi contact:s.niemi@ucl.ac.uk

analysis.analyseBackgroundSubtraction.simpleAnalytical(value=130, size=(50, 50), readnoise=4.5, gain=3.1, req=1.0)

A simple function to test the area of pixels needed (defined by size) to derive the pixel value to the level of required number of electrons given the readout noise and the gain of the system.

Parameters:

• value (int) – the floor level in electrons to be found [default = 130]
• size (tuple) – area describing the number of pixels available [default = (50, 50)]
• readnoise (float) – readout noise of the full detection chain [default = 4.5]
• gain (float) – gain of the detection system [default = 3.1]
• req (float) – required level to reach in electrons [default = 1]

Returns:

none