[gnuastro-commits] master f3619dd5 57/69: Book: edits in the PSF scripts

[Mohammad Akhlaghi]

branch: master
commit f3619dd5acc56b98b99bda0966d10dede11a83be
Author: Mohammad Akhlaghi <mohammad@akhlaghi.org>
Commit: Mohammad Akhlaghi <mohammad@akhlaghi.org>

    Book: edits in the PSF scripts
    
    Until now, I hadn't had a look at the newly added section describing the
    extended PSF creation scripts, or actually tried them out!
    
    With this commit, some edits were made in the book to help in
    readability. In particular, the long paragraphs were shortened and in the
    introduction, instead of simply using the script names, we now have a
    '@ref' to their respective section (that also includes their names).
    
    I also noticed a small issue in the 'psf-create-select-stars' script that
    was due to the new change in behavior for '--noblank' (which now operates
    before column arithmetic). So it was changed to '--noblankend'.
    
    In the tutorial, I also removed the numbers as variables that were only
    used once. Indeed, for a script it is necessary to have these as variables,
    but for a tutorial, they just furstrate the reader who is tyring to make
    sense of everything. Its easier to actually write the value in the text,
    give context, and describe why that choice was made, and use that value in
    the command.
    
    Also, I have re-formatted the outputs of the star stamps phase to be in a
    separate directory and have simple names instead of very long names
    containing the RA and Dec of stars! Such names are hard to read, write,
    find or understand!
    
    However, I stopped doing the tutorial because of the star at
    (201.94042,47.75882): it has a very bright neighbor! The select-stars
    script should have rejected this star.
    
    Sepideh, please correct this in the script (or how its called in the
    tutorial) and let me know so I look into the rest of the work in this
    branch.
---
 bin/script/psf-create-select-stars.in |  21 ++---
 doc/gnuastro.texi                     | 143 +++++++++++++++++++---------------
 2 files changed, 91 insertions(+), 73 deletions(-)

diff --git a/bin/script/psf-create-select-stars.in 
b/bin/script/psf-create-select-stars.in
index b7d70e9d..17a0aea4 100755
--- a/bin/script/psf-create-select-stars.in
+++ b/bin/script/psf-create-select-stars.in
@@ -372,8 +372,8 @@ parallax_error=$(echo "$parallaxanderrorcolumn" \
 
 
 
-# Define a temporal directory and thefinal output file
-# ----------------------------------------------------
+# Define a temporary directory and thefinal output file
+# -----------------------------------------------------
 #
 # Construct the temporary directory. If the user does not specify any
 # directory, then a default one with the base name of the input image will
@@ -454,7 +454,7 @@ if [ x"$catalog" != x ]; then
 
         # Stop if the catalog doesn't overlap with the image.
         echo "Image and catalog do not overlap."
-        echo "Please provide an iamge and a catalog which overlap together."
+        echo "Please provide an image and a catalog which overlap together."
         exit 1
     else
 
@@ -470,7 +470,8 @@ if [ x"$catalog" != x ]; then
 
             # At first check and if there are no stars in the range of 
magnitude
             # the script will be stopped and print the error.
-            numstars=$(asttable $catalog_magfiltered -i | grep 'Number of 
rows' | awk '{print $4}')
+            numstars=$(asttable $catalog_magfiltered -i \
+                          | grep 'Number of rows' | awk '{print $4}')
             if [ "$numstars" = 0 ]; then
                 rm -rf $tmpdir
                 echo "There were no stars in magnitude range $minmag to 
$maxmag"
@@ -483,11 +484,13 @@ if [ x"$catalog" != x ]; then
 else
     # Check if the catalog already exist.
     if [ -f $catalog_magfiltered ]; then
-        echo "External Cataloge already exists "
+        echo "Queried Cataloge already exists "
     else
        # Download the catalog with the necessary parameters.
-        astquery $dataset --output=$catalog_magfiltered --overlapwith=$inputs 
--hdu=$hdu \
-                 --column=$columnquery --range=$field,$minmag,$maxmag \
+        astquery $dataset --output=$catalog_magfiltered \
+                --overlapwith=$inputs --hdu=$hdu \
+                 --column=$columnquery \
+                --range=$field,$minmag,$maxmag \
                  --sort=$field $quiet
     fi
 fi
@@ -585,9 +588,9 @@ asttable $circular -c$racolumn,$deccolumn -c$field | while 
read r d mag; do
 
    # Find number of neighborhod for each star.
    asttable $catalog_magfiltered -c$racolumn,$deccolumn -c$field \
-           -c'arith '$racolumn' '$racolumn' '$r' '$d' distance-on-sphere \
+           -c'arith '$racolumn' '$deccolumn' '$r' '$d' distance-on-sphere \
                      set-i i i '$mindistdeg' gt nan where' \
-            --noblank=4 \
+            --noblankend=4 \
            --colinfoinstdout \
             --output=$numberneighbor $quiet
 
diff --git a/doc/gnuastro.texi b/doc/gnuastro.texi
index 0b70db5c..f13bc0ac 100644
--- a/doc/gnuastro.texi
+++ b/doc/gnuastro.texi
@@ -641,7 +641,7 @@ Installed scripts
 * Sort FITS files by night::    Sort many files by date.
 * Generate radial profile::     Radial profile of an object in an image.
 * SAO DS9 region files from table::  Create ds9 region file from a table.
-* PSF construction and correction::
+* PSF construction and correction::  Tools to build an extended PSF from the 
data.
 
 Sort FITS files by night
 
@@ -655,13 +655,14 @@ SAO DS9 region files from table
 
 * Invoking astscript-ds9-region::  How to call astscript-ds9-region
 
-Tutorial on building the extended PSF
+PSF construction and correction
 
-* Invoking astscript-psf-creat-select-stars::
-* Invoking astscript-psf-create-make-stamp::
-* Invoking astscript-psf-create-junction::
-* Invoking astscript-psf-model-flux-factor::
-* Invoking astscript-psf-model-scattered-light::
+* Tutorial on building the extended PSF::  Tutorial with real-world usage of 
scripts
+* Invoking astscript-psf-create-select-stars::  Select good starts within an 
image.
+* Invoking astscript-psf-create-make-stamp::  Make a stamp of each star to 
stack.
+* Invoking astscript-psf-create-junction::  Merge stacks of different regions 
of PSF.
+* Invoking astscript-psf-model-flux-factor::  Calculate factor to scale PSF to 
star.
+* Invoking astscript-psf-model-scattered-light::  Put the PSF in the image to 
subtract.
 
 Library
 
@@ -22797,7 +22798,7 @@ If you do confront such strange errors, please submit a 
bug report so we fix it
 * Sort FITS files by night::    Sort many files by date.
 * Generate radial profile::     Radial profile of an object in an image.
 * SAO DS9 region files from table::  Create ds9 region file from a table.
-* PSF construction and correction::
+* PSF construction and correction::  Tools to build an extended PSF from the 
data.
 @end menu
 
 @node Sort FITS files by night, Generate radial profile, Installed scripts, 
Installed scripts
@@ -23412,47 +23413,64 @@ The role of the PSF is key in many astronomical 
analysis, and consequently, havi
 In many situations the PSF can be obtained by modeling it with analytical 
functions (Gaussian, Moffat, etc.), see @ref{PSF}.
 However, in other scenarios, it is necessary to obtain an empirical 
(non-parametric) and extended PSF.
 For this reason, here we have developed a set of scripts with the aim of 
constructing the PSF using point-like sources from the astronomical images.
-Once the PSF has been obtained, it can be used for modeling and subtracting 
the point-like sources of astronomical images.
+
 The fundamental ideas of the following methodology are fully described in 
Infante-Sainz et al. (2020, @url{https://arxiv.org/abs/1911.01430}).
-Here we describe the technical implementation of some scripts developed with 
the aim of constructing an empirical PSF, and then, use that PSF for correcting 
the scattered light.
+Once the PSF has been obtained, it can be scaled to the magnitude of the 
point-like sources to subtract.
+Here we introduce several scripts whose ultimately purpose is to do just this.
+We describe the technical implementation of some scripts developed with the 
aim of constructing an empirical PSF, and then, use that PSF for correcting the 
scattered light.
 All scripts are independent of each other, meaning this that the user is free 
to use all of them, or just the ones that are needed.
 
-For constructing the PSF, the basic idea is to consider a bunch of images of 
stars, treat them properly, and finally stack all of them together.
-As a consequence, the first step would be to obtain a catalog of stars.
+For constructing the PSF, the basic idea is to crop images of stars (with 
other sources masked), scale them properly, and finally stack all of them 
together.
+As a consequence, the first step would be to obtain a catalog of stars within 
the datasets.
 In general, not all stars are good to construct a PSF.
-It is desirable to the stars to be free of contamination from other bright and 
nearby objects.
-To select only good candidates we have the script 
@file{astscript-psf-create-select-stars}.
-This script selects only good candidates by different criteria: good parallax, 
not nearby bright stars, axis ratio, etc.
+For example we don't want contamination from other bright, and nearby objects.
+One of the scripts is therefore designed for selecting only good candidate 
stars in your image.
+It will use different criteria, for example good parallax (to avoid confusion 
with galaxies), not near bright stars, axis ratio, etc.
+For more on this script, see @ref{Invoking astscript-psf-create-select-stars}.
 
-Once the catalog of stars is constructed, the script 
@file{astscript-psf-create-make-stamp} is in charge of making appropiate stamps 
of the stars.
+Once the catalog of stars is constructed, another script is in charge of 
making appropiate stamps of the stars.
 Each stamp is a cropped image of the star with the desired size, normalization 
of the flux, and mask of the contaminant objects.
+For more on this script, see @ref{Invoking astscript-psf-create-make-stamp}
 After obtaining a set of star stamps, they can be stacked for obtaining the 
PSF.
 
 Sometimes it is necessary to obtain different regions of the same PSF from 
different stars.
 For example, to construct the far wings it is necessary to consider very 
bright stars.
-However, these stars will be saturated in the most inner part, and 
consequently, fainter stars are necessary.
-In this case, consider using the script 
@file{astscript-psf-create-select-stars} for selecting stars with different 
brightness.
-If different part of the PSF have been constructed, the script 
@file{astscript-psf-create-junction} is in charge of merging the different 
parts together.
+However, these stars will be saturated in the most inner part, and immediately 
outside of the saturation level, they will be deformed due to non-linearity 
effects.
+Consequently, fainter stars are necessary for those regions.
+Therefore, you need to use the two scripts above for selecting stars within a 
fainter magnitude range to build a stack of the inner regions.
+Once different parts of the PSF have been constructed, a third script is in 
charge of merging the different parts together.
+For more on this script, see @ref{Invoking astscript-psf-create-junction}.
+
+Having constructed the PSF as described above (or by any other procedure), it 
can be scaled to the brightness of the various stars in the image (to be later 
subtracted).
+Note that absolute flux of a PSF is meaningless, it should be scaled.
+We therefore have another script that will calculate the scale 
(multiplication) factor of the PSF for each star.
+For more on the scaling script, see @ref{Invoking 
astscript-psf-model-flux-factor}.
 
-Once the PSF has been constructed as described above (or by any other 
procedure), it can be used to model the stars: the scattered light.
-Note that absolute flux of a PSF is meaningless.
-To model a star with a PSF it is necessary to scale the PSF up or down in 
order to put it at the same level of flux than the star.
-To do that, we have the script @file{astscript-psf-model-flux-factor}.
-It is in charge of computing the flux factor by which it is necessary to 
multiply the PSF in order to put it at the same level than a given star.
+Once the flux factor has been computed, a final script is in charge of placing 
the scaled PSF over the proper location in the image and subtract it.
+For more on the scaling and positioning script, see @ref{Invoking 
astscript-psf-model-scattered-light}.
 
-Once the flux factor has been computed, it is possible put the PSF at the same 
sky position and with the same flux level than the star.
-To do that, we have the script @file{astscript-psf-model-scattered-light}.
-It will allocate the PSF at a given position (coordinates) and with the 
appropiate flux level.
+Before going into the technical details of each script, we'll start with a 
tutorial that will use a real-world example to use the scripts in @ref{Tutorial 
on building the extended PSF}.
 
+@menu
+* Tutorial on building the extended PSF::  Tutorial with real-world usage of 
scripts
+* Invoking astscript-psf-create-select-stars::  Select good starts within an 
image.
+* Invoking astscript-psf-create-make-stamp::  Make a stamp of each star to 
stack.
+* Invoking astscript-psf-create-junction::  Merge stacks of different regions 
of PSF.
+* Invoking astscript-psf-model-flux-factor::  Calculate factor to scale PSF to 
star.
+* Invoking astscript-psf-model-scattered-light::  Put the PSF in the image to 
subtract.
+@end menu
 
+@node Tutorial on building the extended PSF, Invoking 
astscript-psf-create-select-stars, PSF construction and correction, PSF 
construction and correction
 @subsection Tutorial on building the extended PSF
-A fully working template example on how to use the scripts in order to 
construct a PSF and then model the scattered light field is described in this 
section.
+In @ref{PSF construction and correction}, a short review of how to use the 
separate scripts in sequence was presented.
+Before digging into the details of each script, we'll start here with a fully 
working template example on how to use the scripts in order to construct a PSF 
and then model the scattered light field is described in this section.
+For this particular example, a J-PLUS image of the galaxy M51 in the filter 
rSDSS is considered.
+For more information about the J-PLUS survey visit: @url{http://www.j-plus.es/}
 
 In this first block set of command lines, necessary variables and parameters 
are defined.
-They are intendeed to have them through the entire process.
 Variables like the number of stars, the size of the stamps, the normalization 
radii, the brightness of the stars, and the url from where the input image is 
downloaded are set here.
-For this particular example, a J-PLUS image of the galaxy M51 in the filter 
rSDSS is considered.
-For more information about the J-PLUS survey visit: @url{http://www.j-plus.es/}
+They will be used during the following steps and are placed here to help in 
customization and readability.
+You can simply copy/paste them all into the terminal that you are running this 
tutorial in.
 
 @example
 $ NSTARS=5
@@ -23460,42 +23478,49 @@ $ STAMPWIDTH=150
 $ NORMRADII_INNER=20
 $ NORMRADII_OUTER=30
 $ IMAGE=image.fits.fz
-$ MINIMUM_MAGNITUDE=0
-$ MAXIMUM_MAGNITUDE=10
-$ MINIMUM_AXISRATIO=0.9
-$ MINIMUM_DISTANCE=0.1
-$ MATCH_APERTURE=0.25
 $ IMAGEID="jplus-dr2/get_fits?id=67510"
 $ URL="http://archive.cefca.es/catalogues/vo/siap/";
 @end example
 
 Once these variables have been set, download the image from the J-PLUS webpage 
using @code{wget}.
-After that, use the script @file{astscript-psf-create-select-stars} to select 
only appropiate stars with the desired brightness.
+After that, open the image with SAO DS9 to have a feeling of the field we are 
using and enjoy the beauty of M51.
 
 @example
 $ wget $URL$IMAGEID -O $IMAGE
+$ ds9 $IMAGE -zoom to fit -zscale
+@end example
 
+@noindent
+First we will build the outer parts of the PSF.
+To do that, we will use @file{astscript-psf-create-select-stars} to select the 
brightest stars in the image (with a magnitude between 6 to 10).
+
+@example
 $ astscript-psf-create-select-stars $IMAGE --hdu=1 \
-      --magnituderange=$MINIMUM_MAGNITUDE,$MAXIMUM_MAGNITUDE \
-      --minaxisratio=$MINIMUM_AXISRATIO \
-      --mindistdeg=$MINIMUM_DISTANCE \
-      --matchaperturedeg=$MATCH_APERTURE \
-      --output=catalog.fits
+                --magnituderange=6,10 \
+                --minaxisratio=0.9 \
+                --mindistdeg=0.1 \
+                --matchaperturedeg=0.25 \
+                --output=catalog.fits
 @end example
 
-Now that the input image and the catalog of the stars have been obtained, it 
is time to construct the individual stamps for each star of the catalog.
-To do that, use the script @file{astscript-psf-create-make-stamp}.
+Now that the input image and the catalog of the stars is ready, it is time to 
construct the individual stamps for each star of the catalog.
+To do that, we'll use @file{astscript-psf-create-make-stamp}.
 Recall that the parameters used for this script have been defined above 
(normalization ring, size of the stamps, etc.).
 Since there are several stars, we iterate over the catalog.
 
 @example
-$ asttable catalog.fits | while read -r ra dec mag; do
-      astscript-psf-create-make-stamp $IMAGE \
-          --mode=wcs \
-          --center=$ra,$dec \
-          --stampwidth=$STAMPWIDTH \
-          --normradii=$NORMRADII_INNER,$NORMRADII_OUTER \
-          --output=starstamp-"$ra"_"$dec".fits; done
+$ counter=1
+$ mkdir stamps-outer
+$ asttable catalog.fits \
+           | while read -r ra dec mag; do
+               astscript-psf-create-make-stamp $IMAGE \
+                    --mode=wcs \
+                    --center=$ra,$dec \
+                    --stampwidth=$STAMPWIDTH \
+                    --normradii=$NORMRADII_INNER,$NORMRADII_OUTER \
+                    --output=stamps-outer/$counter.fits;
+               counter=$((counter+1))
+             done
 @end example
 
 After the stamps are created, they are stacked together with a simple 
Arithmetic command.
@@ -23570,23 +23595,13 @@ Note also that during this process, any mask of 
contaminant sources have been ap
 This tutorial is intendeed to be a basic example from which the reader can 
start to construct the PSF.
 To obtain better results it is necessary to study in details the peculiarities 
of the datasets and consider other options that here they have not been used.
 
-
 In what follows, all options of each script are described in detail.
 For more on installed scripts please see (see @ref{Installed scripts}).
 
-@menu
-* Invoking astscript-psf-creat-select-stars::
-* Invoking astscript-psf-create-make-stamp::
-* Invoking astscript-psf-create-junction::
-* Invoking astscript-psf-model-flux-factor::
-* Invoking astscript-psf-model-scattered-light::
-@end menu
-
-
 
 
-@node Invoking astscript-psf-creat-select-stars, Invoking 
astscript-psf-create-make-stamp, PSF construction and correction, PSF 
construction and correction
-@subsection Invoking astscript-psf-creat-select-stars
+@node Invoking astscript-psf-create-select-stars, Invoking 
astscript-psf-create-make-stamp, Tutorial on building the extended PSF, PSF 
construction and correction
+@subsection Invoking astscript-psf-create-select-stars
 This installed script will select good star candidates for constructing a PSF.
 It will consider stars within a given range of brightness without nearby 
contaminant objects.
 To do that, it allows to the user to specify different options.
@@ -23707,7 +23722,7 @@ The output name of the select stars catalog.
 
 @end table
 
-@node Invoking astscript-psf-create-make-stamp, Invoking 
astscript-psf-create-junction, Invoking astscript-psf-creat-select-stars, PSF 
construction and correction
+@node Invoking astscript-psf-create-make-stamp, Invoking 
astscript-psf-create-junction, Invoking astscript-psf-create-select-stars, PSF 
construction and correction
 @subsection Invoking astscript-psf-create-make-stamp
 This installed script will read an image, the center of an object 
@option{--center}, the type of the center coordinates @option{--mode} (img or 
wcs), the normalization radii @option{--normradii}, and the size of the output 
stamp @option{--stampwidth}.
 With that input, it will generate a stamp centered at the coordinates 
provided, and normalized by the flux computed in the normalization radius.