Re: [PATCH] dmd: Allow storing early logs before writing to disk

[Ludovic Courtès]

David Michael <address@hidden> skribis:

> On Thu, Sep 11, 2014 at 10:31 AM, Ludovic Courtès <address@hidden> wrote:
>> David Michael <address@hidden> skribis:
>>
>>> When using dmd to bring up a read-only file system, it will quit when it
>>> fails to open a log file for writing.
>>
>> Out of curiosity, are you trying to use dmd in the initrd, or maybe on
>> GNU/Hurd?
>
> I've replaced Hurd's /libexec/runsystem script with dmd; i.e., Hurd's
> own init program is launching dmd.  There are some pending patches to
> restructure the Hurd system startup, so I'll probably try to use dmd
> as the real init process once those are accepted.

Nice!

>>> This is a proof-of-concept patch that adds the option to start writing
>>> logs to a string port.  It allows having a dmdconf.scm that runs fsck,
>>> makes the disk writable, and then starts writing past and future log
>>> messages to disk with (start-logging "/var/log/dmd.log").
>>
>> That sounds useful.
>>
>>> Does anyone have any thoughts on this?  Is there a better way to handle
>>> this case?
>>
>> The problem is: when does it figure out that it can now write to the
>> file?
>
> This patch expects that dmd is explicitly told when it's safe to write
> the file.  (See reasoning/concerns below.)  A dmdconf.scm could do
> something like this on boot with it.
>
>     ;; The system was booted read-only ...
>     (system* "/sbin/fsck" "--preen" "--writable")
>     ;; Now it's writable (on success), so dump saved log lines to disk
>     (start-logging "/var/log/dmd.log")

OK.

>> The ideal thing would be:
>>
>>   1. Run ‘dmd -l foo.log’.
>>   2. If foo.log is not writable, then make ‘log-output-port’ a string
>>      port.
>
> Do you think it makes sense to define log-output-port as a string port
> at first instead of a void port?

No, because running dmd without ‘-l’ means disabling logging altogether,
hence the void port.

>>   3. When foo.log becomes writable, have ‘log-output-port’ point to it
>>      and dump previously buffered data.
>>
>> But #3 is difficult.
>>
>> Maybe instead of using a string port, we could use a string port that
>> keeps trying to open the log file?
>>
>> It would be best to use inotify (or the Hurd’s fs_notify), of course.
>
> Something like that would be a nice transparent solution for read-only
> booting, but I could see another issue arising from it later.  It's
> common to have /var or /var/log on a different volume that may not be
> mounted when the root file system's /var/log is writable.  Writing
> logs as soon as possible could miss the sysadmin's desired log volume
> (which theoretically could also happen with dmd as is).

Right, that makes sense.  I think it’s a good argument against trying to
do something too smart.

So probably the patch you propose is the best approach.

Then I would suggest a small change: instead of the magic
‘--log-file=delayed’, what about adding a new option, say,
‘--buffered-log’?  WDYT?

> I haven't done the research, but I'd imagine if other init systems
> write anything to persistent storage, it would be done after fscking
> and/or mounting all relevant entries in /etc/fstab.  It doesn't look
> like dmd is currently doing any /etc/fstab processing, so I've
> included some of that in my dmdconf.scm, and the explicit procedure
> call to start logging declares that the file systems are now in a
> usable state.  Admittedly, it might not be necessary to worry about
> mounting volumes when you have passive translators, but it may be
> cause for concern if the same functionality is used with Linux.

Yeah passive translators greatly simplify things, at the expense of
making it more difficult to track changes to the system configuration.

For the Linux-based distro we don’t use dmd in the initrd.  This is
unfortunate because that leads to some code duplication between initrd
vs. full system, but OTOH, there would be complications with having dmd
do the switch-root thing, I think.

Thanks,
Ludo’.