RE: [Ltib] 3 ways to get uncompressed image size(s)..

[Stephens, Peter A.]

Thanks for the patience guys.  When people ask me, "how big will the
image be ..." they look at me with dismay when I ask them, "What
packages do you need?".  Most of the time they don't know because they
haven't thought that far ahead (conscientious, up-front, systems work is
rare).  

So, I want a way to build an image, determine the size, and list the
packages.  That way I can do this for several configurations that seem
to make sense and give them these numbers with the configurations.  LTIB
can list the packages, so all I need now is the size(s).  I don't mind
using the build numbers, but I want to understand them so I have been
doing this investigation.  

My biggest concern at this point is that the build numbers don't match
what I get when I decompress the image, mount it, and use "du -cks"
(which matches "df") on it.  The build numbers are about 150% of what I
get with these methods.  Really not that bad for an estimate, but why
the large difference?  Is it because the "l" option of "du" counts hard
links multiple times?

Another concern is that some of the man pages and docs (mostly html
files) don't seem to have been stripped.  I thought I saw a
configuration setting that indicated it would delete these, but perhaps
I misinterpreted it, or there are packages that don't honor this yet
(gtk and libxml2?), etc...  Anyway, I was able to reduce the image size
to about half by deleting these.

By "default", I meant that I just took what the configuration was when I
installed LTIB.  I am not really good enough with small Linux systems to
know what is a "minimal" build yet.  That will probably be the subject
of my next research topic, but "minimal" is relative anyway (back to the
need for conscientious, up-front, systems work).

Peter A. Stephens
Advanced Engineering
address@hidden
www.delphi.com
765-451-7074

-----Original Message-----
From: Stuart Hughes [mailto:address@hidden 
Sent: Monday, March 17, 2008 1:24 PM
To: Stephens, Peter A.
Cc: address@hidden; address@hidden
Subject: Re: [Ltib] 3 ways to get uncompressed image size(s)..

Hi Peter,

What default do you mean, I can see gtk in your output, which is not
very minimal.

Padding is default by zero, it can be added in the deployment section
when you configure.  There is a small amount of extra space allocated on
the filesystem depending on the size, this is needed to make sure there
is space for system created logs/files etc.  Take a look at
bin/Ltibutils.pm:sub mk_fs_image for the details, it looks like this:

    my $fs_count = 0;
    find( sub { $fs_count++ }, $stage);
    my ($fs_size) =  split(/\s+/, `LANG=C du -slk $stage`);
    $fs_size += $fs_size >= 20000 ? 16384 : 2400;
    $fs_size += $pcf->{DEPLOYMENT_PADDING_KB}
if$pcf->{DEPLOYMENT_PADDING_KB};

Don't try to calculate the size from the rootfs directory as this
included un-stripped and files that get removed by default (man pages
etc).  As you can see in ltib, it calculates it size by running
  $ du -slk rootfs.tmp
after it has stripped out unwanted files/directories.

What are you trying to achieve? if you want a really small rootfs, you
need to cut back your packages and if it is appropriate select a uClibc
toolchain.  Images under 10MB can benefit greatly by using uClibc,
however be warned that for some platforms this doesn't give the best
optimised performance in some areas (no NPTL, sometimes the math is only
soft etc).

Regards, Stuart



On Mon, 2008-03-17 at 11:55 -0400, Stephens, Peter A. wrote:
> I built an image pretty close to the minimal, default, setup.  I used 
> 3 methods to get the uncompressed sizes.
> 
>     Using "df" after mounting the new file system             32 Mb
> reported
>     Using "du . -ck" after mounting the new file system       32 Mb
reported
>     Using the build output
> 47 Mb reported
> 
> The build output says, "including padding".  What does that mean?  
> What is the padding for?
> Is the padding enough that it makes up the difference between 47.5Mb 
> and the 32Mb reported using the other methods? Can the padding be 
> adjusted or removed?
> 
> Pete
> 
> ================================= Numbers for Minimal image 
> ================================
> 
> -------------------------------- Compressed image file size
> --------------------------------
> 
> ~/ltib> ls -l rootfs.*
> -rw-rw-r-- 1 hzqmrk hzqmrk  8776723 2008-03-14 15:48 rootfs.ext2.gz
> 
> Approximately 8.8Mb.  Pretty good.
> 
> -- Method 1 ----------- Using "df" after mounting the new file system
> -----------------------
> 
> ~/ltib/tmp2> gunzip rootfs.ext2.gz
> ~/ltib/tmp2> sudo mount -o loop rootfs.ext2 /mnt/iso/ ~/ltib/tmp2> cd 
> /mnt/iso/ /mnt/iso> df -l .
> Filesystem 1K-blocks Used  Available Use% Mounted on
> /home/hzqmrk/proj/ltib/tmp2/rootfs.ext2
>            43429     31749 9305  78% /mnt/iso
> 
> OK, so this reports ~ 32 Mb after mounting the image.
> 
> -- Method 2 --------- Using "du . -ck" after mounting the new file 
> system --------------------
> 
> ~/ltib/tmp2> gunzip rootfs.ext2.gz
> ~/ltib/tmp2> sudo mount -o loop rootfs.ext2 /mnt/iso/ ~/ltib/tmp2> cd 
> /mnt/iso/ /mnt/iso> du . -ck | sort -rn | head -n 10
> du: cannot read directory `./lost+found': Permission denied
> du: cannot read directory `./var/run/usb': Permission denied
> 31749   total
> 31749   .
> 20817   ./usr
> 13834   ./usr/share
> 9133    ./usr/share/gtk-doc
> 9132    ./usr/share/gtk-doc/html
> 5033    ./usr/lib
> 5028    ./lib
> 3785    ./usr/share/gtk-doc/html/glib
> 3245    ./usr/share/doc
> 
> I also created a file that contained all the file sizes from the iso 
> using
> 
>       /mnt/iso> find . -type f -name "*" -exec ls -l {} \; > ~/tmp.txt
> 
> and the total size for the files was 29MB.  The difference may be 
> "blocks used" versus "actual" file sizes, but it is so close to the 
> 32MB reported with this method that I think this immaterial.  What is 
> of interest is that I found a lot of documentation in 
> "/usr/share/doc", "/usr/share/gtk-doc", and "/usr/man".  After 
> deleting those, the uncompressed file total was 16.9Mb implying a 
> compressed file of
> (possibly) less than
> 5Mb for this minimal setup.
> 
> -- Method 3 ----------------------- Using build output
> --------------------------------------
> 
> Filesystem stats, including padding:
>     Total size            = 47504k
>     Total number of files = 1102
> Your ramdisk exceeds the old default size of 4096k, you may need to 
> set the command line argument for ramdisk_size in your bootloader 
> allowing 10% free this gives 52254k .  For instance, for u-boot:
> 
> setenv bootargs root=/dev/ram rw ramdisk_size=52254
> 
> creating an ext2 compressed filesystem image: rootfs.ext2.gz
> 
> Started: Fri Mar 14 15:43:44 2008
> Ended:   Fri Mar 14 15:48:18 2008
> Elapsed: 274 seconds
> 
> Build Succeeded
> 
> 
> 
> Peter A. Stephens
> Advanced Engineering
> address@hidden
> www.delphi.com
> 765-451-7074
> 
> -----Original Message-----
> From: Erik Peterson [mailto:address@hidden
> Sent: Friday, March 14, 2008 3:08 PM
> To: Stephens, Peter A.
> Cc: address@hidden; address@hidden
> Subject: Re: [Ltib] Way to calculate uncompressed image size..
> 
> You could always use your cross-compile-"size"
> for example :
> address@hidden bin]$ ./powerpc-linux-size /auto/freescale/ltib/ltib- 
> mpc8544ds-20071122/rootfs/boot/vmlinux
>     text         data     bss     dec     hex filename
> 4004373        198408  191484 4394265  430d19
/auto/freescale/ltib/ltib- 
> mpc8544ds-20071122/rootfs/boot/vmlinux
> 
> 
> On Mar 14, 2008, at 11:17 AM, Stephens, Peter A. wrote:
> 
> >
> > If I want to calculate the uncompressed size of an image can I just 
> > add the packages I want.  Build it.  Change directories to "rootfs" 
> > and run "du . -ckS"?  Here is an example...
> >
> > address@hidden rootfs]$ du . -ckS | sort -rn | head -n 20
> > 118524  total
> > 33896   ./usr/lib
> > 26076   ./usr/lib/gstreamer-0.10
> > 15520   ./usr/bin
> > 6804    ./boot
> > 4904    ./lib
> > 3984    ./usr/share/gtk-doc/html/glib
> > 2676    ./usr/share/gtk-doc/html/libxml2
> > 2660    ./usr/info
> > 2540    ./usr/share/doc/libxml2-2.6.28/html/html
> > 1980    ./usr/share/gtk-doc/html/gobject
> > 1772    ./sbin
> > 1600    ./bin
> > 1252    ./usr/share/sounds/alsa
> > 976     ./usr/share/gtk-doc/html/liboil
> > 808     ./usr/share/man/man3
> > 692     ./usr/share/man/man1
> > 580     ./usr/sbin
> > 528     ./usr/share/doc/libxml2-2.6.28/html
> > 508     ./usr/include/gstreamer-0.10/gst
> >
> > Peter A. Stephens
> > address@hidden <mailto:address@hidden>
> > 765-451-7074
> >
> >
> >
> **********************************************************************
> **
> ****************
> >
> > Note:  If the reader of this message is not the intended recipient, 
> > or an employee or agent responsible for delivering this message to 
> > the intended recipient, you are hereby notified that any 
> > dissemination, distribution or copying of this communication is 
> > strictly prohibited. If you have received this communication in 
> > error, please notify us immediately by replying to the message and 
> > deleting it from your computer. Thank you.
> >
> >
> **********************************************************************
> **
> ****************
> >
> >
> > _______________________________________________
> > LTIB home page: http://bitshrine.org
> >
> > Ltib mailing list
> > address@hidden
> > http://lists.nongnu.org/mailman/listinfo/ltib
> 
> 
> **********************************************************************
> ******************
> 
> Note:  If the reader of this message is not the intended recipient, or
an employee or agent responsible for delivering this message to the
intended recipient, you are hereby notified that any dissemination,
distribution or copying of this communication is strictly prohibited. If
you have received this communication in error, please notify us
immediately by replying to the message and deleting it from your
computer. Thank you. 
> 
> **********************************************************************
> ******************
> 
> 
> _______________________________________________
> LTIB home page: http://bitshrine.org
> 
> Ltib mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/ltib


****************************************************************************************

Note:  If the reader of this message is not the intended recipient, or an 
employee or agent responsible for delivering this message to the intended 
recipient, you are hereby notified that any dissemination, distribution or 
copying of this communication is strictly prohibited. If you have received this 
communication in error, please notify us immediately by replying to the message 
and deleting it from your computer. Thank you. 

****************************************************************************************