Skip to content

Commit

Permalink
Updated/corrected the documentation for wf3rej significantly.
Browse files Browse the repository at this point in the history 
Removed background documentation from wf3rej.py as this text
belongs in the docs.  Updated/added examples for both Python
and C executable usage. Provided explanatory text as to the
actions taken by the underlying program when the options use
their default values.
  • Loading branch information
@mdlpstsci
mdlpstsci committed Apr 17, 2023
1 parent 3ac7f7a commit 493b5d8
Show file tree
Hide file tree
Showing 2 changed files with 114 additions and 108 deletions.
133 changes: 83 additions & 50 deletions docs/wfc3tools/wf3rej.rst
View file
Original file line number Diff line number Diff line change
@@ -1,19 +1,20 @@
.. _wf3rej:

******
wf3rej
******
********
`wf3rej`
********

wf3rej, the cosmic-ray rejection and image combination task in calwf3,
`wf3rej`, the cosmic-ray rejection and image combination task in `calwf3`,
combines CR-SPLIT or REPEAT-OBS exposures into a single image, first
detecting and then replacing flagged pixels. The task uses the same
statistical detection algorithm developed for ACS (acsrej), STIS (ocrrej),
and WFPC2 data (crrej), providing a well-tested and robust procedure.

First, wf3rej temporarily removes the sky background from each input image
(if requested via the SKYSUB parameter in the CRREJTAB), usually computed
using the mode of each image. Sky subtraction is performed before any
statistical checks are made for cosmic rays. Next, wf3rej constructs an
statistical detection algorithm developed for ACS (`acsrej`), STIS (`ocrrej`),
and WFPC2 data (`crrej`), providing a well-tested and robust procedure.

First, `wf3rej` temporarily removes the sky background from each input image
(if requested via the SKYSUB in the CRREJTAB or by a parameter passed
to the Python script or C executable), usually computed using the mathematical
mode of each image. Sky subtraction is performed before any
statistical checks are made for cosmic rays. Next, `wf3rej` constructs an
initial comparison image from each sky-subtracted exposure. This comparison
image can either be a median- or minimum-value sky-subtracted image
constructed from all the input images, and it represents the ‘initial
Expand All @@ -23,7 +24,7 @@ images.

A detection threshold is then calculated for each pixel based on the
comparison image. The actual detection criterion for a cosmic ray is
also calculated. If the etection criterion is greater than the detection
also calculated. If the detection criterion is greater than the detection
threshold, the pixel is flagged as a cosmic ray in the input image’s DQ
array and is ignored when images are summed together. Surrounding pixels
within some expansion radius (CRRADIUS) are marked as ‘SPILL’ pixels and
Expand All @@ -34,13 +35,15 @@ values, computes the true exposure time for that pixel, and scales the sum
to correspond to the total exposure time. The final scaled, cleaned pixel
is written to the comparison image to be used for the next iteration. This
process is then repeated with increasingly stringent detection thresholds,
as specified by CRSIGMAS. See `Section 3.4.5 of the WFC3 Data Handbook <https://hst-docs.stsci.edu/wfc3dhb>`_ for more information.
as specified by CRSIGMAS. See `Section 3.4.5 of the WFC3 Data Handbook
<https://hst-docs.stsci.edu/wfc3dhb/chapter-3-wfc3-data-calibration/3-4-pipeline-tasks>`_
for more information.


Displaying output from wf3rej in a Jupyter Notebook
===================================================
Displaying Output from `wf3rej` in a Jupyter Notebook
=====================================================

When calling `wf3rej` from a Jupyter notebook, informational text output from the underlying `wf3rej.e` program will be passed through `print` as the calibration runs by default, and show up in the user's cell. This behavior can be customized by passing your own function as the `log_func` keyword argument to `wf3rej`. As output is read from the underlying program, the `wf3rej` Python wrapper will call `log_func` with the contents of each line. (`print` is an obvious choice for a log function, but this also provides a way to connect `wf3rej` to the Python logging system by passing the logging.debug function or similar.)
When calling `wf3rej` from a Jupyter notebook, informational text output from the underlying `wf3rej.e` program will be passed through `print` as the calibration runs by default, and show up in the user's cell. This behavior can be customized by passing your own function as the `log_func` keyword argument to `wf3rej`. As output is read from the underlying program, the `wf3rej` Python wrapper will call `log_func` with the contents of each line. Note that `print` is an obvious choice for a log function, but this also provides a way to connect `wf3rej` to the Python logging system by passing the logging.debug function or similar.

If `log_func=None` is passed, informational text output from the underlying program will be ignored, but the program's exit code will still be checked for successful completion.

Expand All @@ -50,20 +53,17 @@ Parameters

input : str or list
Name of input files, such as

* a single filename (``iaa012wdq_raw.fits``)
* a Python list of filenames
* a partial filename with wildcards (``\*raw.fits``)
* filename of an ASN table (``\*asn.fits``)
* a partial filename with wildcards (``*flt.fits``)
* an at-file (``@input``)

output : str, default=""
output : str
Name of the output FITS file.

crrejtab : str, default=""
Reference file name.
Reference filename.

scalense : str, default="" (IS THIS A FLOAT)
scalense : float, default=0.
Scale factor applied to noise.

initgues : str, default=""
Expand All @@ -72,26 +72,26 @@ Parameters
skysub : str, default=""
How to compute the sky (none|mode|mean).

crsigmas : str, default="" (IS THIS A FLOAT)
crsigmas : str, default=""
Rejection levels in each iteration.

crradius : float, default=0
crradius : float, default=0.
Cosmic ray expansion radius in pixels.

crthresh : float, default=0
crthresh : float, default=0.
Rejection propagation threshold.

badinpdq : int, default=0
Data quality flag bits to reject.

crmask : bool, default=False
If True, flag CR in input DQ images.
crmask : bool, default=Setting to be read from CRREJTAB.
If argument is present, write the CR flag value to the input DQ images.

shadcorr : bool, default=False
If True, perform shading shutter correction.
shadcorr : bool, default=Setting to be read from SHADCORR keyword value in primary header of first image to process.
If argument is present, perform shading shutter correction.

verbose : bool, optional, default=False
If True, Print verbose time stamps.
If True, print timestamps and other output.

log_func : func(), default=print()
If not specified, the print function is used for logging to facilitate
Expand All @@ -110,27 +110,60 @@ Usage
.. code-block:: python
from wfc3tools import wf3rej
wf3rej(filename)
from glob import glob
infiles = glob("*flt.fits")
wf3rej(infiles, "output.fits", verbose=True)
wf3rej("*flt.fits", "output.fits", verbose=True)
wf3rej("@input.lst", "output.fits", verbose=True)
Please see the highlighted Note regarding the parameter settings for `wf3rej.e` for more details as to the action taken when the parameters use their default values.

Command Line Options for the wf3rej executable
==============================================
Command Line Options for the `wf3rej` C Executable
==================================================

.. code-block:: shell
wf3rej.e input output [-options]
Input may be a single filename, and the options include:

* -v: verbose
* -t: print the timestamps
* -shadcorr: perform shading shutter correction
* -crmask: flag CR in input DQ images
* -table <filename>: the crrejtab filename
* -scale <number>: scale factor for noise
* -init <med|min>: initial value estimate scheme
* -sky <none|median|mode>: how to compute sky
* -sigmas: rejection levels for each iteration
* -radius <number>: CR expansion radius
* -thresh <number> : rejection propagation threshold
* -pdq <number>: data quality flag bits to reject
wf3rej.e input output [-r] [-v] [-t] [-shadcorr] [-crmask] [-table <filename>]
[-scale <float>] [-init <med|min>] [-sky <none|mode|mean>] [-sigmas <string>]
[-radius <float>] [-thresh <float>] [-pdq <short>]
Example - Process data with timestamps and a custom cosmic ray rejection table:
wf3rej.e ibfma4jqq_flt.fits,ibfma4jtq_flt.fits output.fits -t -table mycrejtab.fits
Example - Print the code version and exit:
wf3rej.e -r
input : comma-separated list of strings
Input filenames as a list of comma-separated input names
ipppssoot_raw.fits,ipppssoot_raw.fits (Note: Do not include any blank spaces.)
output : str
Name of output filename
options
-r : print version number/date of software and exit (no other options selected)
-v : verbose mode
-t : print the timestamps
-shadcorr : perform shading shutter correction
-crmask : set CR flags in input DQ images
-table <filename>: string, the crrejtab filename
-scale <number>: float, scale factor for noise
-init <med|min>: string, initial value estimate scheme
-sky <none|mode|mean>: string, method to compute sky
-sigmas <string of numbers>: string, rejection levels for each iteration (e.g., "3.5,4.5,5.5")
-radius <number>: float, CR expansion radius
-thresh <number> : float, rejection propagation threshold
-pdq <number>: short, data quality flag bits to reject
.. note::

If the ``shadcorr`` option is included on the command line **or** SHADCORR = PERFORM in the primary header of the first image to be processed, the shadcorr correction will be done.

Including the ``crmask`` option on the command line indicates the desire to put the CR flag values into the DQ extension of the input images. Not including this option does *not* turn off the insertion, but rather the program will follow the default setting for the ``crmask`` option as indicated in the CRREJTAB calibration file.

If not all of the following options have been specified on the command line, the CRREJTAB will be read and default values used for the missing options. The options are: crmask, scale, init, sky, sigmas, radius, thresh, and pdq. The CRREJTAB read is either the filename specified by the ``table`` parameter **or** the one specified in primary header of the first image to be processed. In verbose mode, all of the option values are printed to the output logfile.

89 changes: 31 additions & 58 deletions wfc3tools/wf3rej.py
View file
Original file line number Diff line number Diff line change
@@ -1,48 +1,22 @@
"""
wf3rej:
wf3rej, the cosmic-ray rejection and image combination task in calwf3,
combines CR-SPLIT or REPEAT-OBS exposures into a single image, first
detecting and then replacing flagged pixels. The task uses the same
statistical detection algorithm developed for ACS (acsrej), STIS (ocrrej),
and WFPC2 data (crrej), providing a well-tested and robust procedure.
First, wf3rej temporarily removes the sky background from each input image
(if requested via the SKYSUB parameter in the CRREJTAB), usually computed
using the mode of each image. Sky subtraction is performed before any
statistical checks are made for cosmic rays. Next, wf3rej constructs an
initial comparison image from each sky-subtracted exposure. This comparison
image can either be a median- or minimum-value sky-subtracted image
constructed from all the input images, and it represents the ‘initial
guess’ of a cosmic-ray free image. The comparison image serves as the basis
for determining the statistical deviation of each pixel within the input
images.
A detection threshold is then calculated for each pixel based on the
comparison image. The actual detection criterion for a cosmic ray is
also calculated. If the etection criterion is greater than the detection
threshold, the pixel is flagged as a cosmic ray in the input image’s DQ
array and is ignored when images are summed together. Surrounding pixels
within some expansion radius (CRRADIUS) are marked as ‘SPILL’ pixels and
are given less stringent detection thresholds.
Background discussion cn the wf3rej algorithm can be found in the following locations:
https://wfc3tools.readthedocs.io/en/latest/wfc3tools/wf3rej.html,
In summary, the cosmic ray rejection task sums all non-rejected pixel
values, computes the true exposure time for that pixel, and scales the sum
to correspond to the total exposure time. The final scaled, cleaned pixel
is written to the comparison image to be used for the next iteration. This
process is then repeated with increasingly stringent detection thresholds,
as specified by CRSIGMAS. See Section 3.4.5 of the WFC3 Data Handbook for
more information.
The wf3rej executable can also be called directly from the OS command line
prompt:
This routine performs the cosmic ray rejection on input FLT/FLC images and will
produce an output CRJ/CRC image. In contrast to this module, wf3rej.py, which is
a Python wrapper around the C executable, the wf3rej.e C executable can also be
called directly from the OS command line prompt:
>>> wf3rej.e input output [-options]
$ wf3rej.e input output [-options]
Input can be a single file, or a comma-delimited list of files.
Input can be a comma-delimited list of files, and an output file must be specified.
$ wf3rej.e ibfma4jqq_flt.fits,ibfma4jtq_flt.fits output.fits -t -table mycrejtab.fits
Where the OS options include:
Where the C executable options include:
* -r: report version of code and exit (no other options selected)
* -v: verbose
* -t: print the timestamps
* -shadcorr: perform shading shutter correction
Expand All @@ -67,13 +41,11 @@
from .util import error_code


def wf3rej(input, output="", crrejtab="", scalense="", initgues="",
skysub="", crsigmas="", crradius=0, crthresh=0,
def wf3rej(input, output, crrejtab="", scalense=0., initgues="",
skysub="", crsigmas="", crradius=0., crthresh=0.,
badinpdq=0, crmask=False, shadcorr=False, verbose=False,
log_func=print):
"""
Call the calwf3.e executable.
wf3rej, the cosmic-ray rejection and image combination task in calwf3,
combines CR-SPLIT or REPEAT-OBS exposures into a single image, first
detecting and then replacing flagged pixels.
Expand All @@ -89,19 +61,17 @@ def wf3rej(input, output="", crrejtab="", scalense="", initgues="",
----------
input : str or list
Name of input files, such as
- a single filename (``iaa012wdq_raw.fits``)
- a Python list of filenames
- a partial filename with wildcards (``*raw.fits``)
- filename of an ASN table (``*asn.fits``)
- a partial filename with wildcards (``*flt.fits``)
- an at-file (``@input``)
output : str, default=""
output : str
Name of the output FITS file.
crrejtab : str, default=""
Reference file name.
scalense : str, default="" (IS THIS A FLOAT)
scalense : float, default=0.
Scale factor applied to noise.
initgues : str, default=""
Expand All @@ -113,17 +83,20 @@ def wf3rej(input, output="", crrejtab="", scalense="", initgues="",
crsigmas : str, default="" (IS THIS A FLOAT)
Rejection levels in each iteration.
crradius : float, default=0
crradius : float, default=0.
Cosmic ray expansion radius in pixels.
crthresh : float, default=0
crthresh : float, default=0.
Rejection propagation threshold.
badinpdq : int, default=0
Data quality flag bits to reject.
crmask : bool, default=False
If True, flag CR in input DQ images.
If True, flag CR in DQ extension of the input images. If False, the wf3rej
program will read the value of crmask from the CRREJTAB file and follow the
specification. If True in the file, flag CR in DQ extensions of the input
images. If False, do NOT flag CR in the DQ extension of input images.
shadcorr : bool, default=False
If True, perform shading shutter correction.
Expand All @@ -142,27 +115,27 @@ def wf3rej(input, output="", crrejtab="", scalense="", initgues="",
Examples
--------
>>> from wfc3tools import wf3rej
>>> filename = '/path/to/some/wfc3/image.fits'
>>> wf3rej(filename)
>>> from glob import glob
>>> infiles = glob("*flt.fits")
>>> wf3rej(infiles, "output.fits", verbose=True)
>>> wf3rej("*flt.fits", "output.fits", verbose=True)
>>> wf3rej("@input.lst", "output.fits", verbose=True)
"""

call_list = ["wf3rej.e"]
return_code = None

infiles, dummy = parseinput.parseinput(input)
if "_asn" in input:
raise IOError("wf3rej does not accept association tables")
if len(parseinput.irafglob(input)) == 0:
raise IOError("No valid image specified")
if len(parseinput.irafglob(input)) > 1:
raise IOError("wf3rej can only accept 1 file for"
"input at a time: {0}".format(infiles))

for image in infiles:
if not os.path.exists(image):
raise IOError("Input file not found: {0}".format(image))

# Generate a comma-separated string of the input filenames
input = ','.join(infiles)

call_list.append(input)

if output:
Expand Down

0 comments on commit 493b5d8

Please sign in to comment.