Model Data Tuning

In the ideal case, well calibrated models are ready to be used on real data. In the real world, instrument distortions, mask position uncertainty and other instrument effects affect the data in slightly different ways, producing slightly different distortions on a night by night basis. For this reason models must be adjusted on real data, this is what we call models data tuning.

Unlike before, now the computations of the spectra edges as well the line positions for the wavelength calibration, are automatically performed on real data and no more on the base of region files defined by the user.

Since we are working only with dispersed data, this kind of tuning can not be performed on the Optical Model, but only on Curvature Model and Inverse Dispersion Solution Model.

In these section we will see how SpectraPy performs this tuning on real data.

Curvature Model data tuning

The accurate calibration of the spectra curvatures is performed by SpectraPy fitting the edges of the spectra on the frames. These are the main steps of the edges computing:

  1. SpectraPy uses, as first guesses for the edge location, the solution of Curvature Model obtained by the Models calibration procedures

  2. then it moves along this curve, cutting thumbnails containing the edge of the spectrum (blue boxes in the figure)

  3. the library collapses these thumbnails along the dispersion direction and computes edge profile (the red curve in the figure) of each thumbnail

  4. these edge values are used by SpectraPy to fit the model


The TraceCalibration class is the object used by SpectraPy, to fit the edge of the spectra trace

>>> mods_mask = "examples/data/mods1r/ID532016.mdf"
>>> mods1r = "conf/instruments/mods_G670L.icf"

>>> mods_flat = "examples/data/mods1r/mods1r.20180121.0067.fits.bz2"

>>> opt = "examples/data/mods1r/models/MODS1R.opt"
>>> crv = "examples/data/mods1r/models/MODS1R.crv"

>>> from spectrapy.datacalib.tracingcalib import TraceCalibration
>>> trace_calib = TraceCalibration(mods1r, mods_mask, opt, crv)

Now we must define the limits of the tracing computations, and the sizes of the thumbnails

>>> # Define the trace limits
>>> trace_calib.set_trace_limits(1500, 3000)

>>> # Load the image
>>> trace_calib.load_image(mods_flat)

>>> # Define the thumbnails sizes and perform the computation
>>> trace_calib.compute_spectra_traces(slit_win=20, pix_bin=50, var=False)

In this example we selected a range of 1500 pixels in the blue and 3000 in the red. We decided to cut thumbnails every 50 pixels along the dispersion direction (the pix_bin parameter) and each thumbnail is 20 pixels large (along the cross dispersion direction). For each thumbnail the library creates the profile and computes the edge of this profile.


For visual check, SpectraPy displays in DS9 the computed edges. It places green crosses on the computed edge positions. It also shows red crosses in case of failures.

Like we did during Models calibration, in case we are satisfied by the results, we can use these positions to recompute the CRV model (the red crosses will be excluded from computation)

>>> trace_calib.fit_crv_model()

check the result

>>> trace_calib.plot_crv_model(50)

and save the new model

>>> trace_calib.writeto("examples/tmp/", overwrite=True)


In case your instrument is stable enough, i.e. through slits flats match science data, we suggest to use through slits flats to compute spectra curvatures, because the spectra edges are sharper with respect to science data

The longslit case

Like we did during Models calibration, in the longslit case we want to fit both edges. This is done using the right flag in the compute_spectra_traces call.

>>> luci_mask = "conf/masks/luci_LS_0.75.mdf"
>>> luci = "conf/instruments/luci_G200LoRes_1.93_1.8.icf"
>>> sc_file = "examples/data/luci1LoRes/luci1.20180202.0181.fits.bz2"

>>> opt = "examples/tmp/LUCI1.opt"
>>> crv = "examples/tmp/LUCI1.crv"

>>> from spectrapy.datacalib.tracingcalib import TraceCalibration
>>> trace_calib = TraceCalibration(luci, luci_mask, opt, crv)

>>> trace_calib.set_trace_limits(950, 1000)

>>> # We are using a science frame
>>> trace_calib.load_image(sc_file)

>>> # The right flag is now set True
>>> trace_calib.compute_spectra_traces(slit_win=20, pix_bin=50, right=True)

In this example, we also show how to to use a science frame to fit the trace edges instead of trough slit flat.

>>> trace_calib.fit_crv_model()

Like in the MOS case we can check the results and save them

>>> trace_calib.plot_crv_model(100, pos=(0, 0.325, 1))
>>> trace_calib.writeto("examples/tmp/", overwrite=True)


The right=True flag, could be used also in the MOS case. It could be useful when we are working with crowded masks, where slits are very close one to the other, and the edges are not well defined. The right flag, doubles the traces (for each slit we have 2 edges now), increasing the model constraints, and this could help the fitting procedure.

Inverse Dispersion Solution data tuning

The wavelength solution is tuned on real data, searching the real line positions on frames.

The Optical Model gives us the slit extension on the frame along the cross dispersion direction, i.e. for each slit we know how large is the 2D spectrum.

To compute the wavelength solution, SpectraPy cuts N slices of 2D spectrum: one slice for each pixel along the cross dispersion direction. Namely, if the 2D spectrum is 23 pixels large, SpectraPy will create 23 slice of the 2D spectrum. SpectraPy can move along the slice following the Curvature Model Solution and it handles each slice like a 1D spectrum. Moving along the slice it computes the real line positions (relying on one line catalog). Then it computes the slice wavelength solution using these measured lines positions.

In details, for each slice:
  • SpectraPy picks nominal line positions from the input catalog

  • It uses the Inverse Dispersion Solution Model solution (coming from Models calibration) as first guess to move on the expected line position

  • It measures the real line position.

  • for each slice the library refits the 1d polynomial using these real positions.

The WavelengthCalibration class, is the tool used to achieve this calibration. The final product of this procedure is a structure called Extraction Table, saved as FITS table

>>> # The last CRV Module
>>> crv = "examples/tmp/"

>>> opt = "examples/data/mods1r/models/MODS1R.opt"
>>> mods_mask = "examples/data/mods1r/ID532016.mdf"
>>> mods1r = "conf/instruments/mods_G670L.icf"

>>> # The first guess
>>> ids = "examples/data/mods1r/models/MODS1R.ids"

>>> from spectrapy.datacalib.wavelengthcalib import WavelengthCalibration
>>> wave_calib = WavelengthCalibration(mods1r, mods_mask, opt, crv, ids)

Now we must define the range of lambda we want to calibrate (in Angstrom)

>>> wave_calib.set_lambda_range(5000., 9000.)

and load a line catalog. Since in the MOS example we are using an arc frame as calibrator, we select the proper line catalog

>>> mods_arc = "examples/data/mods1r/mods1r.20180121.0073.fits.bz2"
>>> wave_calib.load_image(mods_arc)
>>> NeHg_cat="conf/catalogs/NeHg_hr.dat"

>>> # Do the computation
>>> ID532016_exr = wave_calib.compute_spectra_wave(NeHg_cat)

During this computation we will see appear crosses in the DS9 viewer on the measured lines positions: one cross for each line, each slit and each slice.



Since this is a completely automatic process, only reliable catalog lines, i.e. lines with flag 1 (see Catalog section for details) will be used.

In case we are satisfied by the results, we can save it

>>> ID532016_exr.writeto("examples/tmp/ID532016.exr", overwrite=True)

The longslit case

For longslit data, the procedure is the same. We just reports the list of commands, but there are not differences compared to the MOS case.

>>> luci_mask = "conf/masks/luci_LS_0.75.mdf"
>>> luci = "conf/instruments/luci_G200LoRes_1.93_1.8.icf"
>>> sc_file = "examples/data/luci1LoRes/luci1.20180202.0181.fits.bz2"

>>> crv = "examples/tmp/"

>>> opt = "examples/tmp/LUCI1.opt"
>>> ids = "examples/tmp/LUCI1.ids"

>>> from spectrapy.datacalib.wavelengthcalib import WavelengthCalibration
>>> wave_calib = WavelengthCalibration(luci, luci_mask, opt, crv, ids)
>>> wave_calib.set_lambda_range(15000., 22000.)
>>> wave_calib.load_image(sc_file)
>>> sky_cat = "conf/catalogs/sky_lr.dat"

>>> LS075_exr = wave_calib.compute_spectra_wave(sky_cat)

and save the extraction table

>>> LS075_exr.writeto("examples/tmp/LS075.exr",overwrite=True)


The Extraction Table contains the solution for a given mask, i.e. it is no more a generic model like OPTModel, CRVModel or IDSModel. It contains the 3 models applied to the current mask. If we want to extract spectra acquired with the same instrument configuration, but with another mask, we can tune the valid models on our new data, and we must create a new Extraction Table.