Re: [bug-gnuastro] Error: interpolate_close_neighbors_on_thread

[Mohammad Akhlaghi]

Hi Alejandro,

Thank you very much for the message. I am very happy that Gnuastro is 
being useful for your work. Besides bugs/crashes, please continue to 
share your thoughts (for example suggested improvements) on its programs 
so we can implement them and make it more useful for your work and 
others in the next versions.

This particular crash is due to the fact that NoiseChisel is not able to 
find enough blank regions in the dataset to calculate the quantile 
threshold. With the default settings, it needs at least 9 tiles over the 
dataset where there is no significant signal in the smoothed/convolved 
image (see P.S.). However, as the error message says, with this dataset 
and NoiseChisel configuration, only 4 tiles could be found. You can see 
the successful (non-NaN) tiles with the following command (note the 
`--checkqthresh' option). After running it, a new multi-extension FITS 
image will be created which shows the tiles that are be used):

$ astnoisechisel -h0 ib5x2byrq_00i_single_ext.fits --checkqthresh

The main reason that NoiseChisel can only find 4 tiles is that this 
image has a lot of cosmic rays and the threshold is defined and applied 
on the smoothed image. As you see in the smoothed image (first image 
extension in the output of the command above), these cosmic rays get 
smoothed (and thus cover/remove the regions of good noise).

Since cosmic rays are essentially random, the best way to avoid them, in 
estimating the quantile threshold, is to slightly decrease the tile size 
with this command:

$ astnoisechisel -h0 ib5x2byrq_00i_single_ext.fits --tilesize=40,40

With this command, my NoiseChisel (version 0.4.32) successfully 
completed on this image. The changes in NoiseChisel since your version 
(0.3.53) do not concern this particular problem, so this fix should also 
work on your system too. However, please see P.Ss.

By decreasing the tile size, we are increasing the number of tiles and 
thus decrease the chances that cosmic rays fall in them. But we don't 
want to decrease the tile size too much because their area (number of 
pixels) will decrease significantly and thus the scatter in measuring 
the different parameters (quantile threshold, Sky value and etc) will 
increase, thus degrading the final result. You can see the tiles over 
the input image with the `--checktiles' option.

Another way to fix the problem would be to mask (set to NaN) the cosmic 
rays, but that involves major pre-processing before NoiseChisel with 
other exposures.

I hope this helps,
Cheers,
Mohammad




P.S.

To better understand this, please see the description of `--qthresh' in 
the Gnuastro book:

https://www.gnu.org/software/gnuastro/manual/html_node/Detection-options.html
(or this command: $ info gnuastro "Detection options" )

In particular, the "Quantifying signal in a tile" section fully 
describes how the tiles with no major signal are found:

https://www.gnu.org/software/gnuastro/manual/html_node/Quantifying-signal-in-a-tile.html
(or this command: $ info gnuastro Quantifying )




P.Ss.

Gnuastro version 0.4 was released in September with many bug fixes and 
additions. I recommend subscribing to the info-gnuastro mailing list to 
be informed on every release:

https://lists.gnu.org/mailman/listinfo/info-gnuastro

If you want an even more recent version of Gnuastro, you can always use 
this link to download the most recent un-official version (0.4.32 
currently, the version will be updated in the future):

http://akhlaghi.org/gnuastro-latest.tar.lz

In any case, please consult the `NEWS' file before starting to use 
Gnuastro. In particular as you will see there, since version 0.4, 
NoiseChisel has undergone significant updates (improvements!).