[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Gnuastro 0.18.71 released [alpha]
From: |
Mohammad Akhlaghi |
Subject: |
Gnuastro 0.18.71 released [alpha] |
Date: |
Sun, 9 Oct 2022 19:13:02 +0200 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.3.2 |
Dear all,
The last few months (since the release of Gnuastro 0.18) have been
very productive in Gnuastro (16866 lines added and 4864 lines removed,
in 71 commits) with some pretty exciting, and long-awaited new
features now available (for the full list, see [1] below).
- Warp: align images with the WCS (usually RA/Dec), or another image
(with any rotation/projection/distortion), while accounting for all
WCSLIB-recognized distortions and projections (for the full list of
projection and distortion models see [1] below). In other words, we
are finally free from the chains of the TPV and can directly align
images with SIP or other distortions in creating aligned coadds!
Tip: if you get unreasonable results, upgrade to WCSLIB 7.12.
- ConvertType: draw vector graphics markers based on input catalogs
in the output PDFs (very good for producing high-quality figures in
your papers).
- Statistics: least squares fitting (currently in 1D).
In fact, Pedram (author of the newly added Warp feature above) has
recorded a video highlighing the first two points in Warp and
ConvertType, please watch it to get a hands-on feeling of the power of
these new capabilities using SDSS and J-PLUS images:
https://peertube.stream/w/uq7SBDYZS1HRtJwCkbcDsz
If you have any questions or ideas, feel free to get in touch with
Pedram and the rest of the users and developers in our Matrix chat:
https://openastronomy.element.io/#/room/#gnuastro:openastronomy.org
As always, complete tutorials and documentation for all these features
is also available. Currently the documentation is only in 'info'
format, installed with Gnuastro (the webpage will be updated with
version 0.19). A PDF copy of the latest documentation is available
here: https://akhlaghi.org/gnuastro.pdf
We are therefore almost ready for the next official release of
Gnuastro; hence the reason for this alpha release. It would be great
if you could try building and using this release to help us fix any
important bugs, crashes, compiler warnings and etc. Your feedback will
allow Gnuastro 0.19 to be smoothly usable for all of us :-).
Here is the compressed source and the GPG detached signature for
this release. To uncompress Lzip tarballs, see [2]. To check the
validity of the tarballs using the GPG detached signature (*.sig)
see [3]:
https://alpha.gnu.org/gnu/gnuastro/gnuastro-0.18.71-c982.tar.lz
https://alpha.gnu.org/gnu/gnuastro/gnuastro-0.18.71-c982.tar.lz.sig
Here are the SHA1 and SHA256 checksums (other ways to check if the
tarball you download is what we distributed). Just note that the
SHA256 checksum is base64 encoded, instead of the hexadecimal encoding
that most checksum tools default to (see [4] on how to generate it).
For the list of software used to bootstrap this tarball, see [5].
67df5849a8717da55b73982eaa72082fc9769296 gnuastro-0.18.71-c982.tar.lz
Qw8bU2sZDVR28q8ZaNppFKJUQ0Dgqa106nzUIq25y0c gnuastro-0.18.71-c982.tar.lz
For their direct contribution to this version's source code, I am very
grateful to Pedram Ashofteh-Ardakani (6 commits), Raul Infante-Sainz
(6), Sepideh Eskandarlou (5), Jash Shah (3), Marjan Akbari (2), Elham
Saremi (1) and Faezeh Bijarchian (1). I am also grateful to (in
alphabetical order) Marjan Akbari, Faezeh Bijarchian, Sepideh
Eskandarlou, Giulia Golini, Raul Infante-Sainz, Teet Kuutma and
Richard Stallman for their good suggestions or reported bugs that have
been implemented in this release.
If any of Gnuastro's programs or libraries are useful in your work,
please cite _and_ acknowledge them. For citation and acknowledgment
guidelines, run the relevant programs with a `--cite' option (it can
be different for different programs, so run it for all the programs
you use). Citations _and_ acknowledgments are vital for the continued
work on Gnuastro, so please don't forget to support us by doing so.
Best wishes,
Mohammad
--
Staff Researcher
Centro de Estudios de Física del Cosmos de Aragón (CEFCA),
Plaza San Juan 1, Planta 2, Teruel 44001, Spain
* Noteworthy changes in release 0.18.71
** New features
Book:
- Two new sections added to the "General program usage tutorial" for
describing how to prepare a FITS image for high quality publication (in
PDF), and drawing vector graphics marks from a catalog over it (for
example to show your selected galaxies in the field of view). The use
the newly added features of ConvertType.
Arithmetic:
- Added new type operators and physical constants. All are also available
in Table's column arithmetic also).
- e: Base of the natural logarithm (no units).
- pi: Fraction of Circle cirumference to diameter (no units).
- c: speed of light in vaccume (in units of m/s).
- G: Gravitational constant (in units of m^3/kg/s^2).
- h: Plank's constant (in units of J/Hz).
- au: Astronomical Units (in units of meters).
- ly: Light years (in units of meters).
- avogadro: Avogadro's constant (in units of 1/mol).
- fine-structure: Fine structure constant (no units).
- counts-to-sb: convert counts to surface brightness (mag/arcsec^2).
- sb-to-counts: convert surface brightness (mag/arcsec^2) to counts.
- mag-to-sb: convert magnitudes to surface brightness over a
certain area.
- sb-to-mag: convert surface brightness to magnitudes over a
certain area.
- New operators that are specific to Arithmetic:
- collapse-median: collapse input dataset by calculating the median
along the given dimension.
- collapse-sigclip-std: Collapse with sigma-clipped standard deviation.
- collapse-sigclip-mean: Collapse with sigma-clipped mean.
- collapse-sigclip-median: Collapse with sigma-clipped median.
- collapse-sigclip-number: Collapse with number remaining after
sigma-clip.
ConvertType:
- It is now possible to draw vector graphics marks from a catalog over
the output PDF images. The following options have been added to
ConvertType for doing this. See the "General program usage tutorial"
for a fully working example.
--marks: name of table containing mark information.
--markshdu: HDU of table if file given to '--marks' is FITS.
--markcoords: name or number of two columns containing coordinates.
--mode: if the coordinates are in 'img' (image) or 'wcs' (RA/Dec).
--markshape: name or number of column containing the shape of each
mark.
--markrotate: name or number of column containing rotation of each
mark.
--marksize: name or number of column containing the size of the mark.
--sizeinpix: interpret the values in the size column as pixels.
--sizeinarcsec: interpret the values in the size column as arc-seconds.
--sizeinarcmin: interpret the values in the size column as arc-minutes.
--marklinewidth: name or number of column containing mark's line width.
--markcolor: name or number of column containing mark's color.
--listcolors: List all the 140 available colors, and show the colors on
24-bit (true color) terminal.
--marktext: name or number of column containing text under each mark.
--marktextprecision: number of decimals to print as text when the text
column (given to '--marktext') is floating point.
--markfont: name or number of column containing the font to use for the
the mark text (given to '--marktext').
--markfontsize: name or number of column containing the size of the
font to use for the mark text (given to '--marktext').
--showfonts: build a demo PDF with one page per font to show the
various available fonts on the system.
--listfonts: List the names of the available fonts on the terminal.
- It is now possible to select the color of the border of images produced
in vector graphics outputs (EPS and PDF) with '--bordercolor'.
Fits:
--pixelareaarcsec2: print the image pixel area in units of arcsec^2 to
standard output. Among other things, this is useful in creating a
surface brightness image using the new 'counts-to-sb' operator of
Arithmetic.
Statistics:
- Linear and Polynomial least squares fitting are now available and very
easy to call on the command-line. They are wrappers over the respective
least squares fitting functions of the GNU Scientific Library. The
interface is pretty simple, like the example below:
aststatistics table.fits -cX,Y,Yerr --fit=linear-weighted
It is also possible to estimate values and errors of the fitted model
on a new X axis. A complete example has been added to the newly added
"Least squares fitting" section of the book (under the Statistics
program documentation). Please see that tutorial to easily use this
feature. The following new options have been added to the Statistics
program for this purpose:
--fit: the model to use. Currently the following models are supported:
linear
linear-weighted
linear-no-constant
linear-no-constant-weighted
polynomial
polynomial-weighted
polynomial-robust
--fitweight: nature of the "weight" column (default: standard dev).
--fitmaxpower: maximum power of X in polynomial models.
--fitrobust: weight function to use in the "robust" polynomial model.
--fitestimate: File name, or number to estimate the fit on.
--fitestimatehdu: HDU containing table in file given to
'--fitestimate'.
--fitestimatecol: Column containing X axis values for '--fitestimate'.
Warp:
- Can correct distortions (with any standard recognized by WCSLIB) and
simultaneously align the image to the coordinate system. When no named
linear operation (like '--rotate', '--scale' or etc) is requested, Warp
will go into this mode. It is highly customizable through the following
options. See the "Invoking Warp" section of the book for more. This
feature has been written by Pedram Ashofteh Ardakani.
--center: RA, DEC of the center of the central pixel of output.
--width: Width of output in degrees or pixels (see '--widthinpix').
--widthinpix: interpret values of '--width' as pixels.
--cdelt: Pixel scale of output ('CDELTi' keywords in FITS).
--ctype: Coordinates and projection algorithm. Default: RA/Dec and
Gnomonic or 'TAN').
--edgesampling: extra sampling of pixel polygon to account for strong
non-linear projection or distortion effects, when necessary.
--gridfile: warp the input to the exact WCS and pixel grid of the file
given to this option. This is very useful when matching images from
differetn surveys. Using this option, you can also insert distortions
in an image (for example on a mock image, to make it match an
observed exposure with dithering+distortion).
--gridhdu: HDU containing image to be matched in '--gridfile'.
- List of WCS projections available in WCSLIB 7.12 (and therefore in
Gnuastro's Warp):
AZP: Zenithal/azimuthal perspective.
SZP: Slant zenithal perspective.
TAN: Gnomonic (tangential).
STG: Stereographic.
SIN: Orthographic/synthesis.
ARC: Zenithal/azimuthal equidistant.
ZPN: Zenithal/azimuthal polynomial.
ZEA: Zenithal/azimuthal equal area.
AIR: Airy.
CYP: Cylindrical perspective.
CEA: Cylindrical equal area.
CAR: Plate carree.
MER: Mercator.
SFL: Sanson-Flamsteed.
PAR: Parabolic.
MOL: Mollweide.
AIT: Hammer-Aitoff.
COP: Conic perspective.
COE: Conic equal area.
COD: Conic equidistant.
COO: Conic orthomorphic.
BON: Bonne.
PCO: Polyconic.
TSC: Tangential spherical cube.
CSC: COBE spherical cube.
QSC: Quadrilateralized spherical cube.
HPX: HEALPix.
XPH: HEALPix polar, aka "butterfly".
- List of WCS distortions available in WCSLIB 7.12 (and therefore in
Gnuastro's Warp):
TPD: Template Polynomial Distortion.
SIP: Simple Imaging Polynomial.
TPV: Polynomial distortion for Gnomonic (TAN) projection.
DSS: Digitized Sky Survey.
WAT: Chebyshev or Legendre polynomials for TNX and ZPX projections.
astscript-fits-view:
--ds9colorbarmulti: show a separate color-bar for each image in DS9. By
default this script will show a single color-bar for all the images to
help save space on the monitor when there are many images.
astscript-psf-stamp:
- sub-pixel warping is applied to ensure that your coordinate is at the
center of the central pixel of the output image. This results in a
_major_ improvement when estimating the center of the PSF.
--nocentering: disable sub-pixel warping when creating the PSF stamp. As
described above, the sub-pixel warping is critical for the central part
of the PSF, but for the outer parts it is statistically negligible. So
to avoid slowing down you pipeline, you can disable sub-pixel warping
with this option.
--snthresh: if given, the value to this option is assumed to be a
signal-to-noise ratio (S/N) threshold and all pixels below that S/N
will be masked. This is useful because we are often forced to stack
stars of differing magnitudes. While the fainter ones are good for the
inner parts of the star, they degrade the stack's outer parts. With
this option, the fainter stars won't harm the outer parts.
GNU Make extensions (in a Makefile)
It is now possible to use custom Gnuastro functions in GNU Make, using
its extension facilities with Dynamic libraries. GNU Make is a very
powerful workflow manager that is also used for data analysis (not just
for compilation). With the Gnaustro Make functions, (astronomical) data
analysis becomes even more easier and faster. In the following, you can
see the first set of such functions (they all begin with 'ast-'). For
more, see the newly added chapter in the Gnuastro manual.
- ast-version-is: will return '1' if the running Gnuastro has the given
version (argument of this function). This can be used to ensure
reproducibility in combination with Make's conditional features, see
the minimal working example in the manual.
- ast-text-contains: will return space-separated words within a larger
list that contain a certain string. The to-contain string can be
anywhere within the words of the larger list.
- ast-text-not-contains: will return space-separated words within a
larger list that DO NOT contain a certain string. The to-not-contain
string can be anywhere within the words of the larger list.
- ast-fits-with-keyvalue: takes a keyword name, a list of keyword
values, a HDU and a list of FITS files. It will return only those
FITS files that have the requested value(s) in the requested keyword
of the requested HDU.
- ast-fits-unique-keyvalues: takes a keyword name, a HDU and a list of
FITS files. It will return all the unique values given to that
keyword within the FITS files.
Library:
- GAL_CONFIG_HAVE_PYTHON: non-zero if Python3+Numpy features included.
- GAL_CONFIG_HAVE_GNUMAKE_H: non-zero if GNU Make extensions can be made.
- gal_box_border_rotate_around_center: width of box after rotation.
- gal_color_id_to_name: return the name of a color from its ID.
- gal_color_in_rgb: return the fraction of red-green-blue in a color.
- gal_color_name_to_id: return the ID of a color from its name.
- gal_dimension_collapse_median: collapse input along dim. using median.
- gal_dimension_collapse_sigclip_mean: collapse with sig-clipped mean.
- gal_dimension_collapse_sigclip_std: collapse with sig-clipped STD.
- gal_dimension_collapse_sigclip_median: collapse with sig-clipped
median.
- gal_dimension_collapse_sigclip_number: collapse with num. after
sig-clip.
- gal_eps_shape_id_to_name: return the name of a shape from its ID.
- gal_eps_shape_name_to_id: return the ID of a shape from its name.
- gal_fit_name_to_id: Convert string name to ID of fitting model.
- gal_fit_name_from_id: Convert ID of fitting model to string name.
- gal_fit_name_robust_to_id: Convert name of robust weights to ID.
- gal_fit_name_robust_from_id: Convert ID of robust weights to name.
- gal_fit_1d_linear: linear fit of input columns.
- gal_fit_1d_linear_no_constant: linear fit with no constant.
- gal_fit_1d_linear_estimate: estimate a linear fit on a new X column.
- gal_fit_1d_polynomial: polynomial fit of input columns.
- gal_fit_1d_polynomial_robust: robust polynomial fit of input columns.
- gal_fit_1d_polynomial_estimate: estimate a polynomial fit on new X
column.
- gal_fits_unique_keyvalues: extract all unique values to a certain
keyword in many files.
- gal_fits_with_keyvalue: select FITS image with a certain key value.
- gal_list_data_select_by_name: select a dataset from a list by its name.
- gal_list_str_cat: Concatenate (append) list to a space-separated
string.
- gal_list_str_extract: Extract space-separated tokens to a list.
- gal_python_type_from_numpy: Convert Numpy's type id. to Gnuastro's.
- gal_python_type_to_numpy: Convert Gnuastro's type id. to Numpy's.
- gal_txt_contains_string: Check a certain string within in a larger one.
- gal_units_counts_to_sb: SB from counts, zeropoint and area.
- gal_units_mag_to_sb: surface brightness (SB) from magnitude and area.
- gal_units_sb_to_counts: counts from SB, zeropoint and area.
- gal_units_sb_to_mag: magnitude from SB and area.
- gal_warp_wcsalign_init: initialize the WCS aligning structure.
- gal_warp_wcsalign_onpix: Per-pixel filling of output.
- gal_warp_wcsalign_onthread: function to give to pthreads.
- gal_warp_wcsalign: high-level function to align input by its WCS.
- gal_warp_wcsalign_free: free the contents of the WCS aligning
structure.
- gal_wcs_free: free a WCS structure that is created or read by Gnuastro.
** Removed features
Statistics:
--refcol has been removed because it breaks the modularity principle
(given that it is the job of Gnuastro's Table program to limit rows
from a larger table based on many different criteria). The output of
Table can be directly piped to Statistics to achieve the same (and much
more feature-rich effect).
Warp:
--align: has been removed. This is because aligning an image (while
correcting for any possible distortion) is now the default behavior of
Warp (when no linear operations have been requested).
** Changed features
Book:
- The "General program usage tutorial" section is now the first section
of the Tutorial chapter, since it introduces the tools at a more basic
level. The "Sufi simulates a detection" (which was previously first)
has been moved to the fourth section.
Warp:
- The short format of the '--centeroncorner' option has been removed. The
'-c' is now the short format for the new '--center' option to Warp.
astscript-psf-scale-factor:
--widthinpix: new name for the old '--stampwidth' option. This was done
to have the same name to a similar option in Crop and help in
remembering.
astscript-psf-stamp:
--widthinpix: new name for the old '--stampwidth' option. This was done
to have the same name to a similar option in Crop and help in
remembering.
Library
- gal_eps_write: two new arguments have been added to draw marks, and to
set the border color.
- gal_pdf_write: similar to 'gal_eps_write'.
- gal_fits_hdu_open: new argument to optionally exit program if HDU
couldn't be opened.
** Bugs fixed
bug #62861: '--printkeynames' of Fits program gets caught in an infinite
loop on FITS files that have empty keywords before
'END'. Found by Pedram Ashofteh Ardakani.
bug #62892: Installed scripts don't account for differing LANG and
LC_NUMERIC. Found by Teet Kuutma and fixed by Raul
Infante-Sainz.
bug #62937: psf-scale-factor not being quiet, when requested. Found and
fixed by Sepideh Eskandarlou.
bug #62943: Couldn't read the value of width with '--snthresh' is called.
Found and fixed by Sepideh Eskandarlou.
bug #62944: No warning when the option value isn't immediately after the
equal sign in long format. Found by Faezeh Bijarchian.
bug #63013: Sigma clip segmentation fault when input has an integer type
with values close to saturation-level.
bug #63022: psf-scale-factor not saving the result in the output file,
found and fixed by Raul Infante-Sainz.
[2] Lzip has better compression ratio and archival features compared
to the `.gz' or `.xz' formats. Therefore Gnuastro's alpha/test
releases are only in this format, but for historical reasons we also
include `.gz' tarballs in the official releases. If you don't have
Lzip (you can check with `lzip --version' command), download and
install it from its webpage:
https://www.nongnu.org/lzip/lzip.html
If Lzip is present and you use GNU Tar, then the single command below
should uncompress and un-pack the tarball:
$ tar xf gnuastro-0.18.71-c982.tar.lz
If the command above doesn't work, you have to un-compress and un-pack
it with two separate commands (or use a pipe to feed the output of the
first into the second: `lzip -cd gnuastro-0.18.71-c982.tar.lz | tar -xf -'):
$ lzip -d gnuastro-0.18.71-c982.tar.lz
$ tar xf gnuastro-0.18.71-c982.tar
[3] Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact. First, be sure to download both the .sig file
and the corresponding tarball. Then, run a command like this:
gpg --verify gnuastro-0.18.71-c982.tar.lz.sig
The signature should match the fingerprint of the following key:
pub rsa4096 2018-12-08 [SC]
52B0 4484 D806 C90D CB52 7249 71E8 9901 2D17 4B66
uid [ unknown] Mohammad Akhlaghi <mohammad@akhlaghi.org>
If that command fails because you don't have the required public key,
or that public key has expired, try the following commands to retrieve
or refresh it, and then rerun the 'gpg --verify' command.
gpg --recv-keys 71E899012D174B66
As a last resort to find the key, you can try the official GNU
keyring:
wget -q https://ftp.gnu.org/gnu/gnu-keyring.gpg
gpg --keyring gnu-keyring.gpg --verify gnuastro-0.18.71-c982.tar.lz.sig
[4] To get the base64 SHA256 checksum, you can use the command below
(assuming you already have the 'openssl' command-line program). Just
note that an extra '=' is printed as the last character that you
should ignore.
cat gnuastro-0.18.71-c982.tar.lz \
| openssl dgst -binary -sha256 | openssl base64 -A
[5] This tarball was bootstrapped (created) with the tools below. Note
that you don't need these to build Gnuastro from the tarball, these
are the tools that were used to make the tarball itself. They are only
mentioned here to be able to reproduce/recreate this tarball later.
Texinfo 6.8
Autoconf 2.71
Automake 1.16.4
Help2man 1.49.2
ImageMagick 7.1.0-50
Gnulib v0.1-5527-gcd1fdabe8b
Autoconf archives v2022.09.03-4-gda89908
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- Gnuastro 0.18.71 released [alpha],
Mohammad Akhlaghi <=