On 12/07/2012 08:34 AM, LRN wrote:
And "spare" is the problem. I can easily spare 20 or 40 gigabytes, but
100 or 200 is somewhat trickier. I might have that kind of space now,
and be willing to give it to GNUnet, but i might want that space back
at some point. Not sure what GNUnet will do right now, if i shut down
my node, reduce the datastore size, then start the node up again.
Probably discard lowest-priority blocks until datastore shrinks to the
new limit?
Yes, that's what it would do. However, there is a caveat: the
mysql/sqlite/postgres database that is involved might be happy to delete
the records, but might not automatically reduce its file system space
consumption. So you may have to additionally trigger some
database-specific routine to force the database to defragment/relinquish
its allocation/garbage collect/whatever.
Doing this may (temporarily) double your space requirements, depending
on the database. So this is an "implementation detail" that would make
an automatic 'shrink if disk is full' implementation somewhat harder
(but likely not impossible, as you can predict the necessary space for
the reorganization). Alternatively, one _may_ be able to use multiple
database files and just delete one of those entirely once the quota is
reached (this depends on the database backend that is being used).
Now, having a minimum space allocated to the datastore, and then just
using N% of the remaining free disk space for for datastore too, while
it's available - that really makes the decision easier. If GNUnet is
then taught to use pre-allocated datastore for important blocks (files
being downloaded or published; what are privacy issues here?), that
would mean that your node will serve _your_ interests first, and will
use the free space available to serve the network as best as it can.
I don't think there is a problem here. We already have routines to
shrink-to-quota which are triggered if we are above quota (due to
additional insertions or due to quota being lowered).
It should maintain either F% of space free, or G gigabytes (whichever
is larger). Obviously, F and G are configurable (i.d say - default F
to 20, and G to 20; unless GNUnet daemon that would reclaim free space
would be a slowpoke, 20 gigabytes should give it enough time to react).
It should also be completely disabled for SSDs, IMO. Because they are
small to begin with, _and_ because their performance degrades greatly
as they are filled with data.
I suspect those arguments may not hold for long as SSD technology
progresses...
Thus the idea is the same as with CPU resources - you set up low and
high thresholds for CPU load that GNUnet can cause. It will go as high
as the high threshold when uncontested, and will go down to the low
threshold when other processes compete for CPU resources with GNUnet.
Same for storage - use large portion of available free space for
datastore (primarily - for migrated and cached blocks), but be ready
to discard all that, and go as low as the size of the pre-allocated
datastore.
Well, LRN, if you think peers actually run close to quota, there is a
nice GNUNET_UTIL-call for starters: GNUNET_DISK_get_blocks_available.
Adjusting the quota option in datastore based on that should not be too
hard for you; the real bitch will be testing the various backends to
make sure that they actually reduce disk space consumption --- and I
guess reliably finding out which partition MySQL/Postgres actually store
their data on might also be not so easy...
Happy hacking! ;-).
-Christian
_______________________________________________
GNUnet-developers mailing list
address@hidden
https://lists.gnu.org/mailman/listinfo/gnunet-developers