[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[gnuastro-commits] master 0d5776a 1/2: Pointer library: memory-mapped fi
From: |
Mohammad Akhlaghi |
Subject: |
[gnuastro-commits] master 0d5776a 1/2: Pointer library: memory-mapped files only written in .gnuastro_mmap |
Date: |
Sun, 15 Mar 2020 15:44:59 -0400 (EDT) |
branch: master
commit 0d5776acd448de897ab08aaf1acc5041a9fbaf3f
Author: Mohammad Akhlaghi <address@hidden>
Commit: Mohammad Akhlaghi <address@hidden>
Pointer library: memory-mapped files only written in .gnuastro_mmap
Until now, when a file needed to be memory-mapped, Gnuastro's programs
(through the `pointer.h' library) would first try writing the mmap files in
the `.gnuastro' directory. When it failed it would attempt writing in the
`.gnuastro_mmap' directory. However, `.gnuastro' is also used to store
configuration files (which are hand-written and thus valuable). Mixing the
two types of source (configuration files) and automatically generated
(memory-mapped) files is very problematic.
With this commit, it no longer attempts to write memory-mapped files under
`.gnuastro'. It will only attempt to write them under the `.gnuastro_mmap'
directory.
---
NEWS | 10 ++++++++++
doc/gnuastro.texi | 13 +++++--------
lib/pointer.c | 22 ++++++++--------------
3 files changed, 23 insertions(+), 22 deletions(-)
diff --git a/NEWS b/NEWS
index c27be58..43c9e97 100644
--- a/NEWS
+++ b/NEWS
@@ -48,6 +48,16 @@ See the end of the file for license conditions.
** Changed features
All programs and libraries:
+ --minmapsize: Gnuastro's programs no longer attempt to write
+ memory-mapped files under `.gnuastro'. They will only attempt to write
+ them under the `.gnuastro_mmap' directory. Until now, when an internal
+ array needed to be memory-mapped, Gnuastro's programs (through the
+ `pointer.h' library) would first try writing the mmap files in the
+ `.gnuastro' directory. When it failed it would attempt writing in the
+ `.gnuastro_mmap' directory. However, `.gnuastro' is also used to store
+ configuration files (which are hand-written and thus valuable). Mixing
+ the two types of source (configuration files) and automatically
+ generated (memory-mapped) files is very problematic.
- FITS ASCII tables: When a column has a floating point type, but its
ASCII string can't be parsed as a number, it will be read as a
NaN. Until now, the corresponding program/library would abort,
diff --git a/doc/gnuastro.texi b/doc/gnuastro.texi
index e6414dd..f388b9d 100644
--- a/doc/gnuastro.texi
+++ b/doc/gnuastro.texi
@@ -6225,9 +6225,8 @@ In such scenarios, without this option, the program will
crash.
A random filename is assigned to the array.
This file will keep the contents of the array as long as it is necessary and
the program will delete it as soon as its not necessary any more.
-If the @file{.gnuastro} directory exists and is writable, then the random file
will be placed in there.
-Otherwise, the @file{.gnuastro_mmap} directory will be checked.
-If @file{.gnuastro_mmap} does not exist, or @file{.gnuastro} is not writable,
the random file will be directly written in the current directory with the
@file{.gnuastro_mmap_} prefix.
+If the @file{.gnuastro_mmap} directory exists and is writable, then the random
file will be placed in there.
+Otherwise, the randomly named file will be directly written in the current
directory with the @file{.gnuastro_mmap_} prefix.
By default, the name of the created file, and its size (in bytes) is printed
by the program when it is created and later, when its deleted/freed.
These messages are useful to the user who has enough RAM, but has forgot to
increase the value to @code{--minmapsize} (this is often the case).
@@ -19261,11 +19260,9 @@ value of this variable is @code{NULL}, then the
contents of @code{array}
are actually stored in RAM, not in a file on the HDD/SSD. See the
description of @code{minmapsize} below for more.
-If a file is used, it will be kept in the hidden @file{.gnuastro}, or
-@file{.gnuastro_mmap} directories with a randomly selected name to allow
-multiple arrays to be kept there at the same time, see description of
-@option{--minmapsize} in @ref{Processing options}. When
-@code{gal_data_free} is called the randomly named file will be deleted.
+If a file is used, it will be kept in the hidden @file{.gnuastro_mmap}
directory.
+Its name is randomly selected to allow multiple arrays at the same time, see
description of @option{--minmapsize} in @ref{Processing options}.
+When @code{gal_data_free} is called the randomly named file will be deleted.
@item size_t minmapsize
The minimum size of an array (in bytes) to store the contents of
diff --git a/lib/pointer.c b/lib/pointer.c
index 58e6d70..e01a2a5 100644
--- a/lib/pointer.c
+++ b/lib/pointer.c
@@ -115,30 +115,24 @@ gal_pointer_allocate_mmap(uint8_t type, size_t size, int
clear,
size_t bsize=size*gal_type_sizeof(type);
- /* Check if the .gnuastro folder exists, write the file there. If it
- doesn't exist, then make the .gnuastro directory. If it can't be
- built, we'll make a randomly named directory. */
- gal_checkset_allocate_copy("./.gnuastro/", &dirname);
+ /* Check if the `.gnuastro_mmap' folder exists, write the file there. If
+ it doesn't exist, then make it. If it can't be built, we'll make a
+ randomly named file in the current directory. */
+ gal_checkset_allocate_copy("./.gnuastro_mmap/", &dirname);
if( gal_checkset_mkdir(dirname) )
{
- /* Free the old name. */
+ /* The directory couldn't be built. Free the old name. */
free(dirname);
- /* Try `.gnuastro_mmap' (to avoid making a separate directory for
- each memory mapping if possible). */
- gal_checkset_allocate_copy("./.gnuastro_mmap/", &dirname);
- if( gal_checkset_mkdir(dirname) )
- {
- free(dirname);
- dirname=NULL;
- }
+ /* Set `dirname' to NULL so it knows not to write in a directory. */
+ dirname=NULL;
}
/* Set the filename. If `dirname' couldn't be allocated, directly make
the memory map file in the current directory (just as a hidden
file). */
- if( asprintf(filename, "%smmap_XXXXXX", dirname?dirname:"./.gnuastro_")<0 )
+ if( asprintf(filename, "%sXXXXXX", dirname?dirname:"./.gnuastro_mmap_")<0 )
error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
if(dirname) free(dirname);