IDL software to simulate MOSES Image Deconvolution

 


This document describes IDL software which can be used to simulate MOSES data and deconvolve the data to a λ,x,y data cube. The very core part of it was written by Charles C. Kankelborg. The principle is to use an existing data cube, e.g. measurements of some target done with CDS or SUMER or artificially created data, covering only one single line (1), calculate the three two dimensionl images MOSES would produce of this target (i.e. the orders +1, -1, and 0), and perform an image reconstruction based on the Fourier Slice Theorem. The theory behind the image deconvolution is described in the document "Deconvolution with MOSES"



Setup


A tarball (size 72 kb) with all routines can be downloaded here:


moses-simulation.tar


The tarball, when unpacked, creates a subdirectory moses-simulation, containing some IDL procedures and functions. The procedure to simulate MOSES data and to do a deconvolution is called sim_deconv.pro. The procedure prepgraphics.pro creates some maps and histogramms in form of arrays packed into a structure and written into a file. the procedure mkgraphics.pro creates some plots based on this maps and histograms, which visualize the quality of the image reconstruction, in particular in reproduced line centers and line widths. The procedure paramtertrend.pro investigates the systematic error occuring in line center reproduction, and suggests some correcting factors (2). All other IDL .pro files are functions called by sim_deconv.pro, prepgraphics.pro or mkgraphicss.pro.


Example data to run sim_deconv on can be downloaded here and should be placed in a subdirectory moses-simulation/datacubes:


cds_ov.sav (516 kb)


The example data have to be three dimensional arrays called cube with the wavelength axis in the 1st dimension.


Another file needed in the moses-simulation/datacubes directory is


DATA


which contains information for each file of example data such as rest wavelength etc.. Further explanations are given in the file itself. If you add your own data cube files, you have to update that file. In the end, you should have the following file structure in your working directory:

cleanup.pro
combine_overlap.pro
contains.pro
datacubes
datacubes/DATA
datacubes/cds_ov.sav
despike.pro
get_background.pro
get_cube.pro
get_lineparameters.pro
graph_convergence.pro
iter_backprj.pro
make_4masks.pro
make_masks.pro
minutes.pro
mkgraphics.pro
nlines.pro
overlappograms4.pro
overlappograms.pro
parametertrend.pro
prepgraphics.pro
sim_deconv.pro
write_results.pro
zeropad.pro


All IDL procedures or functions (.pro files) is documented in their headers (if not, please let me know).



Run a MOSES data and deconvolution simulation


The software has been written under IDL Version 5.5, and makes use of the routine plot_image in solarsoft. After compilation of sim_deconv.pro,


IDL> .compile sim_deconv


run


IDL> sim_deonv, "cds_ov", "test_", 0, 0, 0


The first argument, "cds_ov", is a string determining what data cube in the datacubes subdirectory we simulate on (it is the filename without the .sav extension). The second argument, "test_", is a string which is a prefix to all filenames produced by the procedure. It serves as a later identifier for the simulation. For each two dimensional lambda-x slice of the data cube, the procedure calculates the three spectral orders MOSES would see of the target, calculates the average line profile, replaces each average line profile with a gauss fit, and performs a reconstruction of the slice. The last three parameters, all set to zero in this example, are offsets to add on the line centers, line widths and line intensities of the gauss fits. If set to values different to zero, they simulate a "bad guess" for the average line profiles.


The procedure may run many minutes, depending on the size of the data cube. The fits of the average line profiles are monitored in a graphical window. A second window shows each image slice during the iterative procedure reducing negative intensities. On the terminal window (where the command line was typed in), information on the gauss fits in each slice are given. A third window compares the average spectrum with the actual spectrum it is replaced with. This is basically above fit (displayed in the first window) after normalizing the total intensity to the other three overlappograms. If there are other lines or features in the spectral range, this will lead to an overestimation in the total line intensity! To work more accurate, all features in the average spectrum would have to be 'guessed', but this is probably beyond the scope of what can be done in real life.


The above command produces the following files:


test_cube.sav
test_backcube.sav
test_convergence.eps
test_lineparameters_av.dat
test_totnegs


test_cube.sav is the original data cube, test_backcube.sav the reconstructed data cube. test_lineparameters_av.dat summarizes parameters of the gaussian line profile fits and are needed in later analysis using prepgraphics.pro. test_totnegs contains the total negative intensity of the output slices. test_convergence.eps is a graph demonstrating the convergence for the last image slice.



Analysis of results


prepgraphics.pro makes use of the files produced by sim_deconv.pro and calculates some histogramms and maps. They are stored in a structure as a file. Run


IDL> prepgraphics, "cds_ov", "test_"


Again, the name of the data file and the file prefix are given as arguments. The terminal window displays some statistical information based on the histograms of the error of reconstructed line center, error of corrected reconstructed line center (see footnote 2), and error of reconstructed line width. The individual columns mean minimum value, maximum value, average value, and full width at half maximum of the distribution. It looks somewhat like this:


test_ centerr:  min/max/av/fwhm: -0.637722 0.476559 0.0229063 0.300099 17.1567
test_ corcent:  min/max/av/fwhm: -0.868016 0.814120 0.0213052 0.323194 18.4770
test_ widtherr: min/max/av/fwhm: -0.812526 0.999291 0.0276274 0.601564


Units are pixels. For line centers, the full width at half maximum is also given in km/s (last column). prepgraphics.pro produces a number of files, listed below with their meanings:


test_results.sav		contains a structure with histograms, maps, and values 
			 	for averages and FWHM
test_lineparameters.dat		line fit parameters (for internal use)
test_lineparameters_c.dat	line fit parameters (for internal use)
LOGFILE				is updated each time to summarize all runs of prepgraphics.


The procedure mkgraphics.pro pics up the structure saved in test_results.sav and produces some postscript files:


test_centerhisto.eps		difference of true and reconstructed line center in a histogram
test_centerhisto.dat		histogram data
test_centerhisto_corr.eps	difference of true and corrected reconstructed line center (2)
test_centerhisto_corr.dat	histogram data
test_widthhisto.eps		difference of true and reconstructed line width in a histogram
test_widthhisto.dat		histogram data

test_centermap.eps		map of true (left) and reconstructed (right) line center
test_centermap_corr.eps		map of true (left) and corrected reconstructed (right) line center (2)
test_centerdiffmap.eps		map of line center differences
test_centerdiffmap_c.eps	map of line center differences for corrected line centers (2)

test_widthmap.eps		map of true (left) and reconstructed (right) line widths
test_widthdiffmap.eps		map of line width differences

test_image.eps			map of true (left) and reconstructed (right) intensities for the 
				wavelength point of maximum intensity


For illustration, here are the graphic results for the line center reproduction of this examples, i.e. the files test_centermap.eps, test_centerdiffmap.eps and test_centerhisto.eps (from left to right):



The images should be self-explanatory. Finally, the procedure parametertrend.pro investigates the systematic error seen in reconstructed line centers:


IDL> parametertrend, "test_"


The only neccessary argument is the file prefix string. The procedure plots the difference between the true line center λ (known from the input data cube) and the line center of the average profile of the corresponding image slice, λ_av, vs. the difference between the reconstructed line center λ_re and the true line center λ into a postscript file test_trend.eps. Such a plot looks like this:



The procedure also performs a linear fit to the cloud of points in this diagram:


(λ_re - λ) = A + B(λ - λ_av)


Ideally, the point cloud would not extend along the y-axis at all, meaning that the reconstructed line center agrees for all pixels with the input value. With only random errors resulting from the reconstruction, the point cloud will have some vertical scatter, but in such a sense that a linear fit has still zero slope B and zero point A=0 (i.e. runs horizonally and through the origin). The latter turns out to be true, but the slope is not zero, pointing at a systematic error in a sense that the reconstructed line centers are underestimated and pulled towards the average value. This is a direct consequence of the fact that the data cube reconstruction is performed with only 4 projected images (three spectral orders and average line profile). One can imagine the average line profile to pull the result towards itself, thus causing an underestimation of the individual line centers. It is the average data of such linear fits, performed on many test images, which could provide a correction factor for the line center reproduction. The average fit parameters enter the code in the procedure prepgraphics.pro. Unfortunately, it turnes out that the slope of above fits is not unique for all possible test images but depends on the image itself. That is why the graph test_trend_c.eps, which shows the same as test_trend.eps but with the correction applied, does not show a point cloud with zero slope (see also footnote 2). The correction factor hardwired in prepgraphics.pro will have to be changed when a suitable correction has been found.


For the sake of completeness, the procedure paramtertrend.pro performs the same analysis on the line widths, so that the files created by the procedure are:


test_trend.eps
test_trend_c.eps
test_trendw.eps


Moreover, zero point and slope of the fits are printed on the terminal window:


test_ center -      fit zeropoint, slope:    0.00259538    -0.573666
test_ corr.center - fit zeropoint, slope:     0.0102484    -0.312289


Again, in the ideal case, the numbers in the last row should both be zero.



Miscalleneous


The Perl script eps2jpg transforms encapsulated postscript files to jpeg format and creates an icon sized picture which comes in handy for html documents:


./eps2jpg file.eps


The files created are called file.jpg and file_icon.jpg. The script can also run on a file filelist containing a list of .eps files:


./eps2jpg -file filelist


So a quick way to create jpegs and icons out of all .eps files would be


ls *.eps > temp
./eps2jpg -file temp


The script mkhtml uses the html template restemplate.html and produces a file test_result.html serving as a quick-look html file displaying the most interesting graphs. If more than one test run results should be displayed on the same web page, mkhtml can take a file containing a list of prefixes:


./mkhtml -file list


(1) the software is not designed for multiple emission line spectra, but for single line profiles only. For multiple line spectra, the replacement of the average spectrum by a gaussian profile should be extended to fit all lines. When applying the simulation to a multiple lined spectrum, only one line will be considered and the others will be ignored. This leads to an overestimation of the lines intensity, since the corresponding overlappogram is normalized in respect to the other three (orders 0, 1, and -1) overlappograms.

(2) an average value of such correction factors is already hardcoded in prepgraphics.pro and the reason for output files with the appendix string "_c" or "_corr" in parallel to all other file output. However, a good correction factor has not yet been found, and probably won't be found as a unique factor since it depends somewhat on some global property of the image. How to correct systematic errors in line centers is still being investigated.




Armin Theissen, Jan 15 2004, last update: Feb 06 2004