Release notes for UVOTPY

Notes to version 2.0

Versions 2.0.0 and 2.0.1 had the intial version of the improved treatment of the coincidence loss, but without the fine-tuned coi-area.

In versions 2.0.2 and 2.0.3 the coincidence loss has been treated using the standard theory of coincidence loss corrections with fine tuned coi-area to each grism mode.

Functionality is basically the same as described below for version 1.0, with some bug fixes and updates, many due to the inclusion of increasingly more Astropy software.

Notes to version 1.0

Version 1.0 2013-11-01

General

This code is designed for processing Swift UVOT Grism images

The goal is to process the spectra in the image, to do quality control and source identification. Because the UVOT is a photon counting detector, the error handling must keep track of the errors which require in principle the total exposure, the background exposure, and the exposure time. The data quality for each pixel should be flagged also (e.g., data dropouts, scattered light rings, halo around bright features), but flagging is only done for nearby zeroth orders. While the count rate errors follow from the observed binomial nature of the detector, e.g., Kuin & Rosen (2008,MNRAS, 383,383) - though that prescription was for point sources. A heuristic method was used to develop a correction for coincidence loss used in this program.

Details of the accuracy and reliability of the calibration and software are soon to be submitted to the Monthly Notices of the RAS.

This program extracts the spectra, applies the wavelength calibration file to find anchor position and wavelength dispersion. The flux calibration is valid over the whole detector and depends on the effective area and coincidence loss correction. In both grisms the effective area was determined at several offset positions The accuracy of the flux in the uv grism is of order 5% in the centre and about 10% at other locations on the detector. In the visible grism the accuracy of the flux is 20%, which error is dominated by that in the coincidence loss correction.

In the nominal grism mode (wheelpos = 200, 1000) the response varies by about 5% from centre to about 200 pixels from the edge. In the clocked grism modes (wheelpos = 160, and 955) the response has a strong drop when the spectrum falls in the upper left corner. For the rest of the image the response varies less than ~20% in the clocked modes.

Before using this code, it is recommended to reprocess the raw image using the mod-8 correction. Use a CALDB later than May 2010 for an improved distortion correction. The grism detector image should be attitude corrected using uvotgraspcorr. Check the header keyword ASPCORR=’GRASPCORR’.

Find out the RA and DEC position in decimal degrees from the USNO-B1 catalog for your object since the UVOT aspect corrections also use the USNO-B1 positions, so you will avoid a systematic error in positions which will translate to a shift in the wavelengths derived.

You need to set the environment variable UVOTPY to point to the directory with the UVOTPY code and spectral calibration files.

The input data files may have to be decompressed before running the program, although it will try to do that itself, but does not recompress the files again.

The program was developed running in iPython, and it is suggested to run interactively in iPython rather then run as it this script.

Second order extraction and calibration are only treated very roughly at this time. Zeroth orders are only minimally identified. Third order location is approximate.

Main functions

getSpec : main call for spectral data extraction

Other uvotpy functions

curved_extraction: set curvature of orders, set quality flags, get spectral data extractSpecImg : get sub image findBackground : get background get_components : extract first, second, third order components getCal : get the (wave) calibration files predict_second_order : use first order to predict second order (very rough) coi_func : return an interpolating function(wave) for the coincidence loss of a spectrum [experimental]

Specialized functions are in modules

uvotgetspec: repository of main functions uvotio: writes file output uvotio.rate2flux : convert count rate to flux uvotplot : plot routines uvotmisc : miscellaneous routines

Files

The program assumes all data files are available in either the working directory, or the directory structure complies with the Swift project standard and is run from the <obsid>/uvot/images directory, while the attitude file is available in the <obsid>/aux directory. There is rudimentary support for running from a remote directory on the same device, but the program will write some files to both the current and the data directories.

The flux-calibrated 1st order spectrum is available in the second extension of the output file.

From version 1.0 onwards, the file name includes a flag “_f” for when lenticular filter image(s), or “_g” when “uvotgraspcorr” aspect corrections were used to derive the anchor position. Both methods give similar uncertainties, but for the same field uvotgraspcorr will give more consistent results, while the lenticular filter method works when uvotgraspcorr cannot find an aspect correction (in that case the uncorrected pointing position from the star trackers will be used).

History

2013-Oct-31 Paul Kuin

Rewritten uvotwcs to fix a bug in the calculation of the pointing. Small fixes all throughout. Output file names will have a flag for the anchor point method used.

2013-May-23 Paul Kuin

revised the uv grism clocked mode wavecal for first and second order. Missing still is second order effective area.

March 2013: Flux calibration for the uv-grism

The new calibration file has multiple extensions. The first extension is the original flux calibration, which did not correct for the coincidence loss and which is to be used with the “Ftool” uvotimgrism. Then follow extensions with the measured effective areas at various positions on the detector. These have been corrected for coincidence-loss during the calibration. The number depends to some extent on how much the effective area varies with position. Finally, a normalised flux from a rescaled model is included that can be used to extrapolate from one of the effective areas at a nearby position. A new routine in uvotio reads the effective area and model scaling to offset if present.

A start has been made with using Sphinx (http://sphinx.org) for documentation on the web. This includes changing the inline documentation of the software. Keep an eye on http://ucl.ac.uk/mssl/astro/space_missions/swift.

A preliminary flux calibration for the visual grism has been made. It supersedes the old calibration which did not take any coincidence-loss correction into account. However, future improvements are expected.

October 2012: Initial extension of flux calibration uv-grism

The software has been updated to support a working coi-correction, though not the final one, and use the new flux calibration where available. The calibration file closest to the position on the detector image will be used. More calibration files will be supplied when they become available. The comparison of the new calibration and coi-loss correction with calibration spectra can be seen on my web site.

The package has now been created using Python distutils.

March 2012: An experimental coincidence loss correction

The background in the grism is quite high and in itself experiences about 3% coi-loss. The extended nature of the background has been cause for concern, but tests reported in Breeveld et al. (2010) did in fact show that the correction method from Poole et al. (2008) works well also in that case. However, the case for an extended linear feature, like a spectrum has not

March 2012: An experimental coincidence loss correction

The background in the grism is quite high and in itself experiences about 3% coi-loss. The extended nature of the background has been cause for concern, but tests reported in Breeveld et al. (2010) did in fact show that the correction method from Poole et al. (2008) works well also in that case. However, the case for an extended linear feature, like a spectrum has not been studied. The symmetries are different, and it is not a priori clear how to best generalize the method from Poole et al. But with estimated coi-loss peaking at 3% for a 16th mag WD and estimated 7% for a 14th mag WD, with maximum coi-loss close to 50% for a 12th mag WD, it is clear that a solution is desirable as part of a flux calibration that is meaningfull. The WD are probably the worst case, since they are bright around 3000A where the effective area (and the count rate) peaks

March 2012: discovery of variable sensitivity over face of uv clocked detector image

Over most of the detector, the sensitivity only changes by a few percent. So it was a bit of a shock to see that in the clocked uv grism images, the detector sensitivity drops quite a lot in the upper left corner of the detector. More so, since this area had been selected to place spectra to completely avoid the zeroth orders. This is under investigation. We are using the Zemax optical model to get a quantitative, though approximate, measure of this effect.

April 2012: predicting the uv grism exposure

For some faint objects, there is often uvot photometry in the uv filters available. It is of a given magnitude, but what does that mean for the uv grism. I made a tool that will make a rough estimate of the exposure time needed to get a certain signal-to-noise, assuming a certain background and given a magnitude of the object as observed in a uvot filter. There is loads of uncertainty when the source is faint, and the background over the UV clocked grism varies. It defaults to 0.16c/s/pix(across the spectrum) but a value of 0.05 may happen depending of where the (uv part of) the spectrum lies on the detector. So for now, I put in the most reasonable value. If the source is getting faint, the exposure time will climb through the roof. Experience must tell how good it is and how to use it.

Altermatively a spectrum can be put in a will give a magnitude in the lenticular filters. The uvotphot.py file will be needed.

April 2012: estimate of the zeroth order effective area

The uv grism efficiency in the zeroth first and second order estimate were needed for calculating the flux sensitivity over the detector using the Zemax optical model. A rough approximation of the zeroth order effective area was determined in the course of that work. Details have been written in a report.

April 2012: determining the effective area in the uv detector

Testing

The software was tested with the following installations:

A. dated October 2013 Debian Squeeze4

1. I have a working installation of the HEASARC CALDB and HEASOFT (swift)
2. I installed Ureka as described in Installing UVOTPY with the Ureka distribution.
3. tested using various spectra, both from the command line and interactive.

B. dated March 2013

(1) an existing installation of the STScI python, including the Scipy software, for Python version 2.7.3 A package can be downloaded from STScI. Note that you also will need to install gfortran (link at the bottom of the STScI Python page) to make it work. If you have pyraf installed, it may already be there.

(2) Installation of the Heasoft Ftools/SWIFT software, release not earlier than version 6.10. Download from http://heasarc.gsfc.nasa.gov/docs/software/lheasoft/ .

(3) Installation of the HEASARC CALDB for Swift, version later than Oct 13,2011. Download the CALDB for Swift from: http://heasarc.gsfc.nasa.gov/docs/heasarc/caldb/caldb_supported_missions.html

(4) Installation of Mink’s WCSTOOLS, in particular scat. On unix/linux type system the command which scat should return the location. If it does not return anything, install: http://tdc-www.harvard.edu/software/wcstools/wcstools-3.8.4.tar.gz and remove or rename the program cphead that it installs, since it clashes with cphead in Heasoft as used by the uvotproduct ftool.

(5) Installation of CDSCLIENT which provides the ‘sesame’ name resolver. [optional] http://cdsarc.u-strasbg.fr/doc/cdsclient.html

(6) setup of the environment: (“$HOME/.cshrc” for csh users or “$HOME/.bashrc” for bash or sh users)

• add the directory with this software to the PYTHONPATH environment variable.
• set UVOTPY to the directory with this software
• add the WCSTOOLS/bin and cdsclient directories to your PATH