Ticket #7353 (closed defect: fixed)

Opened 6 years ago

Last modified 6 years ago

olpc3/F9 builds are bloated.

Reported by: cscott Owned by: dgilmore
Priority: normal Milestone: 8.2.0 (was Update.2)
Component: distro Version: Development build as of this date
Keywords: Cc: cscott, jg, mstone, dgilmore, bernie, dsd, mtd, tomeu
Action Needed: never set Verified: no
Deployments affected: Blocked By: #234, #6188, #7367, #7368
Blocking: #5535, #7383, #7416

Description

We added about 60M to the build when shifting from F9. This bug will track efforts to identify where the extra bloat is coming from, and efforts to remove it.

Attachments

pkgsizes.sh (377 bytes) - added by dsd 6 years ago.
hacky script to calculate package sizes
sizes.txt (1.9 kB) - added by dsd 6 years ago.
sizes of new packages, update1 vs 8.2
sizes.2.txt (1.2 kB) - added by dsd 6 years ago.
updated package sizes

Change History

  Changed 6 years ago by cscott

Some initial observations:

  • perl snuck back in
  • lots of icons, for a number of different themes, in a number of formats.
  • xine has plugins for display hardware the XO doesn't have.
  • gnome stock icons re-added; i think we managed to remove them previously?
  • some redhat graphics also found their way back into the build.
  • lots of .pat sound files, where did they come from?
  • /usr/share/firstboot/themes
  • /usr/share/rhgb (which we don't use)
  • lots of hal support for hardware we don't have
  • /usr/share/gnome/help (in lots of languages)
  • did we always install all languages?
  • /usr/share/metacity/icons ?
  • /usr/share/kde-settings ?
  • lvm / mdadm stuff, did we have that before?

  Changed 6 years ago by cscott

  • blockedby 234 added

  Changed 6 years ago by cscott

  • blockedby 6188 added

  Changed 6 years ago by cscott

  • cc dgilmore, bernie added
  • next_action set to never set
  • blockedby 7367, 7368 added
  • perl is due to mtd-utils, ntp, gstreamer-plugins-base, and libbonobo, trac bug #234
  • lots of theme crap (including /usr/share/rhgb and /usr/share/kde-settings) due to libgnome requiring fedora-gnome-theme, trac #7367
  • xine plugins part of xine-lib, /usr/lib/xine/plugins/1.21/vidix takes up 208k. We should either fork these or delete them post-install in pilgrim.
  • the midi patches (54M worth!) are being pulled in by PersonalCopy-Lite-patches, which is providing timidity++-patches, which is required by SDL_mixer, which provides libSDL_mixer-1.2.so.0, which is required by pygame, trac #7368
  • totem requires gnome-desktop, which requires a number of silly things and provides /usr/share/gnome/help and crap. totem also requires gnome-themes, which requires gnome-icon-theme, which we do not need (trac #4259, #4621). Do we need totem?
  • lvm/mdadm is pulled in by mkinitrd, which is pulled in by the kernel, trac #6188.

  Changed 6 years ago by marco

We need totem for the web browser plugin, we used to build a version of it with dependencies "stripped off".

What do we need xine for?

  Changed 6 years ago by cscott

Current size comparison:

  • joyride-2057: 455M
  • joyride-2079: 726M

totem-mozplugin seems to be pulling in *both* totem-xine *and* totem-gstreamer, and totem-xine seems to be pulling in xine. Dennis is going to work on hacking this tangled web apart.

  Changed 6 years ago by cscott

  • blocking 5535 added

(In #5535) I reformatted the ticket description to clean up the wiki markup.

See also #7353 for *more* things to clean up.

In email I've also suggested the following packages from OLPC_DEVEL_PACKAGES (thanks to Bert for the pointer):

http://lists.laptop.org/pipermail/devel/2008-June/015869.html

  • dbench (26M!)
  • pciutils (1.3M)
  • xterm (378k)
  • xorg-x11-apps (259k)
  • xorg-x11-twm (100k, maybe not worth it)

  Changed 6 years ago by cscott

The olpc-logos package in fedora was used in the olpc2 branch to supply redhat-logos and fedora-logos; it probably needs to be re-added & updated:

http://koji.fedoraproject.org/koji/rpminfo?rpmID=203932

  Changed 6 years ago by dsd

  • cc dsd added
  • blocking 7383 added

  Changed 6 years ago by cscott

Also /usr/sbin/dellWirelessCtl, dellBiosUpdate, dellLEDCtl, dellLcdBrightness (!)

Pulled in from smbios-utils, which is required by the hal package. Luckily it's only (!) 300k.

We might consider making a "olpc-kludge" package which reports itself as Providing a number of these crazy things, so we don't have to go through an fork hal, etc.

  Changed 6 years ago by cscott

  • blocking 7416 added

  Changed 6 years ago by cscott

For ntp, the scripts /usr/sbin/ntptrace and /usr/sbin/ntp-wait use perl (the /usr/sbin/ntpd binary matches perl as well but that's a false alarm). I'd suggest splitting the ntp package as other distros do, so we can get ntpdate (which we use) w/o getting the rest of ntpd.

  Changed 6 years ago by cscott

In joyride-2102 we started pulling in an extra 20M of samba stuff via gvfs which totem-gstreamer Requires. New packages:

+bluez-libs 3.32-1.fc9
+fuse 2.7.3-2.fc9
+fuse-libs 2.7.3-2.fc9
+gphoto2 2.4.0-10.fc9
+gvfs 0.2.3-11.fc9
+gvfs-fuse 0.2.3-11.fc9
+libarchive 2.4.17-1.fc9
+libexif 0.6.16-1.fc9
+libsmbclient 3.2.0-2.17.fc9
+libtalloc 1.2.0-17.fc9
+libtdb 1.1.1-17.fc9
+lockdev 1.0.1-12.fc9.1
+samba-common 3.2.0-2.17.fc9
+samba-winbind 3.2.0-2.17.fc9

We did get rid of these, though:

-gnome-desktop 2.22.2-1.fc9
-libXvMC 1.0.4-4.fc9
-libmng 1.0.9-6.1
-libmodplug 1:0.8.4-3.fc9
-libmpcdec 1.2.6-4.fc9
-mesa-libGLU 7.1-0.35.fc9
-perl-Config-IniFiles 2.39-6.fc9
-redhat-menus 8.9.11-3.fc9
-totem-xine 2.23.2-4.fc9
-xine-lib 1.1.12-2.fc9

so we're making some progress...

  Changed 6 years ago by mtd

  • cc mtd added

  Changed 6 years ago by mstone

Seems to me that we could cheat here by letting our package manager install all the RPMs it wants, then force-remove the ones that we want to get rid of. Comments?

  Changed 6 years ago by cscott

mstone: that leaves a 'broken' RPM configuration, so it's a last ditch resort. We're still trying to fix most of these the 'right way', and I think the 'olpc-kludge' package is slightly better than force-remove, since it doesn't affect future use of yum/rpm.

  Changed 6 years ago by dsd

xulrunner grew some deps which I've killed: https://bugzilla.redhat.com/show_bug.cgi?id=454890

also working on killing e-d-s: totem depends on totem-pl-parser (doesn't seem realistic to remove this dep), and pl-parser now uses a single date parsing function from libcamel which ends up pulling the whole whopping e-d-s package in. http://svn.gnome.org/viewvc/totem-pl-parser/trunk/plparse/totem-pl-parser.c?r1=17&r2=18

separating libcamel into it's own package doesn't seem to be that good an option because that lib has a crapload of deps of its own:

	linux-gate.so.1 =>  (0xb7f3c000)
	libedataserver-1.2.so.9 => /usr/lib/libedataserver-1.2.so.9 (0xb7e9e000)
	libssl3.so => /lib/libssl3.so (0xb7e6d000)
	libsmime3.so => /lib/libsmime3.so (0xb7e45000)
	libnss3.so => /lib/libnss3.so (0xb7d0e000)
	libnssutil3.so => /lib/libnssutil3.so (0xb7cf5000)
	libplds4.so => /lib/libplds4.so (0xb7cf2000)
	libplc4.so => /lib/libplc4.so (0xb7cec000)
	libnspr4.so => /lib/libnspr4.so (0xb7cb2000)
	libgnomevfs-2.so.0 => /usr/lib/libgnomevfs-2.so.0 (0xb7c4f000)
	libgconf-2.so.4 => /usr/lib/libgconf-2.so.4 (0xb7c18000)
	libgthread-2.0.so.0 => /lib/libgthread-2.0.so.0 (0xb7c13000)
	librt.so.1 => /lib/librt.so.1 (0xb7c09000)
	libgmodule-2.0.so.0 => /lib/libgmodule-2.0.so.0 (0xb7c04000)
	libgobject-2.0.so.0 => /lib/libgobject-2.0.so.0 (0xb7bc4000)
	libglib-2.0.so.0 => /lib/libglib-2.0.so.0 (0xb7ae2000)
	libpthread.so.0 => /lib/libpthread.so.0 (0xb7ac9000)
	libkrb5.so.3 => /usr/lib/libkrb5.so.3 (0xb7a29000)
	libk5crypto.so.3 => /usr/lib/libk5crypto.so.3 (0xb7a04000)
	libcom_err.so.2 => /lib/libcom_err.so.2 (0xb7a00000)
	libgssapi_krb5.so.2 => /usr/lib/libgssapi_krb5.so.2 (0xb79d2000)
	libsasl2.so.2 => /usr/lib/libsasl2.so.2 (0xb79b9000)
	libssl.so.7 => /lib/libssl.so.7 (0xb796e000)
	libcrypto.so.7 => /lib/libcrypto.so.7 (0xb7820000)
	libdl.so.2 => /lib/libdl.so.2 (0xb781b000)
	libz.so.1 => /lib/libz.so.1 (0xb7806000)
	libc.so.6 => /lib/libc.so.6 (0xb769d000)
	libxml2.so.2 => /usr/lib/libxml2.so.2 (0xb754e000)
	libbonobo-2.so.0 => /usr/lib/libbonobo-2.so.0 (0xb74e9000)
	libbonobo-activation.so.4 => /usr/lib/libbonobo-activation.so.4 (0xb74d3000)
	libORBit-2.so.0 => /usr/lib/libORBit-2.so.0 (0xb7479000)
	libdb-4.6.so => /lib/libdb-4.6.so (0xb7330000)
	libdbus-glib-1.so.2 => /usr/lib/libdbus-glib-1.so.2 (0xb7312000)
	libdbus-1.so.3 => /lib/libdbus-1.so.3 (0xb72d1000)
	libavahi-glib.so.1 => /usr/lib/libavahi-glib.so.1 (0xb72cd000)
	libavahi-common.so.3 => /usr/lib/libavahi-common.so.3 (0xb72c1000)
	libavahi-client.so.3 => /usr/lib/libavahi-client.so.3 (0xb72b0000)
	libresolv.so.2 => /lib/libresolv.so.2 (0xb729b000)
	libselinux.so.1 => /lib/libselinux.so.1 (0xb727f000)
	libutil.so.1 => /lib/libutil.so.1 (0xb727b000)
	/lib/ld-linux.so.2 (0xb7f3d000)
	libkrb5support.so.0 => /usr/lib/libkrb5support.so.0 (0xb7272000)
	libkeyutils.so.1 => /lib/libkeyutils.so.1 (0xb726e000)
	libcrypt.so.1 => /lib/libcrypt.so.1 (0xb723c000)
	libm.so.6 => /lib/libm.so.6 (0xb7213000)
	libORBitCosNaming-2.so.0 => /usr/lib/libORBitCosNaming-2.so.0 (0xb720e000)
	libnsl.so.1 => /lib/libnsl.so.1 (0xb71f3000)
	libcap.so.2 => /lib/libcap.so.2 (0xb71ef000)

I'm going to mail Bastien (totem upstream) and see if he'll reconsider his date parsing approach.

  Changed 6 years ago by dsd

I mailed a patch to Dennis to remove totem-pl-parser's dependency on libcamel. In addition to that and xulrunner I have also worked on the following:

  • We forked gnome-python2 in OLPC-2 to remove some dependencies and we should do so again. Basically gnome-python2 includes libgnome bindings which pull in libgnome (and then loads of other crap including window managers and icon themes). We don't actually use the libgnome bindings, we just use the gnomevfs component. I sent a new spec file to Dennis which is hopefully suitable for Fedora upstream too, and will result in the libgnome portions dropping out of the dependency chain.
  • We forked gnome-vfs2 in OLPC-2 to remove the gnome-mount dependency and should do again. The code fails gracefully when it is asked to mount something and cannot find any utilities to do so.

My work on the above 4 packages should result in (at least) the following packages falling out of the build: gnome-mount, PolicyKit-gnome, libsexy, hunspell-en, hunspell, evolution-data-server, gnome-keyring, libgnome, fedora-gnome-theme, nokoko-metacity-theme, metacity, control-center-fs, fedora-icon-theme, gnome-themes, gnome-python2-bonobo. Possibly libbonobo and libbonoboui too.

  Changed 6 years ago by dsd

I mailed this question to Dennis but should probably ask out in the open as well: How far do we want to go with forking to remove dependencies?

portaudio grew a dependency on jack-audio-connection-kit which pulls in libfreebob. It would be easy to fork portaudio and add the --disable-jack configure option to remove those 2 deps, but they aren't really that big: yum info says jack is 365k and freebob is 434k. Worth it or not?

  Changed 6 years ago by erikos

Wow jack-audio-connection-kit and libfreebob. What does pull in portaudio, csound?

  Changed 6 years ago by dsd

espeak pulls in portaudio, we've shipped portaudio in update1 too (it's 226kb)

  Changed 6 years ago by cscott

  • cc tomeu added

Let's order our work by size removed. The target is 300Mb for our build, we've got 50M to trim. When we run out of bigger things to trim, we should move to ~1M things like portaudio -- but, basically, every fork has a cost. I'd rather fork "only" 10 packages which shave 5M each to meet our size goal than fork 50 packages shaving only 1M each.

So, the short answer for portaudio is probably "not yet".

In the meantime, the sugar packages have grown a dependency on xorg-x11-server-Xephyr which adds 2M. That's a regression, and to a package for which we are the upstream, so we should certain fix that. cc'ing tomeu, not because he's responsible, but because he might know who is.

  Changed 6 years ago by cscott

Xephyr addition was in joyride-2134 / sugar 0.81.6 as shown by http://dev.laptop.org/~bert/joyride-pkgs.html. The packages also dropped their 'olpc3' dist tag, so maybe we're inadvertently not pulling from our olpc3 sugar fork?

  Changed 6 years ago by erikos

Yup will remove it we just think currently about the best way to keep our specs sane.

  Changed 6 years ago by cscott

Re perl & ntp, note that the ntp changelog at http://dev.laptop.org/~rwh/announcer/joyride_vs_update1.html says,

--- Changes for ntp 4.2.4p4-6.fc9 from 4.2.4p2-3.fc7 ---
...
 + require perl (#274771)

That upstream bugzilla reference might be a good place to start in trying to tease perl back out of ntp. Splitting a separate 'ntpdate' packages from ntpd would be a good start, since we only use ntpdate.

  Changed 6 years ago by dsd

Simon released my xulrunner work, and I have pushed public_rpms for gnome-python2, gnome-vfs2 and totem-pl-parser. Hopefully next joyride will be noticably slimmer.

I will work on the ntpdate split now.

Changed 6 years ago by dsd

hacky script to calculate package sizes

Changed 6 years ago by dsd

sizes of new packages, update1 vs 8.2

  Changed 6 years ago by erikos

I did the sugar package (joyride 2175) - so xephyr is out won 2 MB :)

Changed 6 years ago by dsd

updated package sizes

  Changed 6 years ago by dsd

I filed some bugs to get my package dieting merged into fedora:

I also filed some bugs for packages where we should try and convince Fedora upstream to make such modifications to the official Fedora packages before we diverge majorly:

Also on this topic I dropped an experimental pam RPM into my public_rpms which should kill cracklib and its dictionaries (#5259)

  Changed 6 years ago by dsd

As another experiment I have locally created a fork of xorg-x11-utils like we did for OLPC-2, it removes xdriinfo which should kill mesa stuff. We are hopefully at <300mb now.

  Changed 6 years ago by cscott

The joyride-2203 build is at 298M now. Yay! That's our target for 8.2, although we'll probably have to keep trimming to maintain the builds at this size.

Is there anything left in #4281 / #5535 to fix? There's some unfinished work in #1286, for example.

  Changed 6 years ago by dsd

A lot of stuff is coming from my public_rpms, I want to get that in fedora before closing tickets. Dennis sponsored me today (thanks), so I will cautiously work on doing that.

  Changed 6 years ago by dsd

I've pushed all my previous work into Fedora, not sure if anyone wants to push further on #4281 / #5535? I'm planning to raise #1286 in today's meeting, as it will cause more activity breakage...

follow-up: ↓ 34   Changed 6 years ago by cscott

Joyride-2231 is at 300M exactly. However, the latest sugar-evince added poppler (3M) -- I wonder if we can get that back out to give ourselves a bit of margin?

in reply to: ↑ 33   Changed 6 years ago by cjb

Replying to cscott:

Joyride-2231 is at 300M exactly. However, the latest sugar-evince added poppler (3M) -- I wonder if we can get that back out to give ourselves a bit of margin?

<marcopg_> c_scott: poppler is needed

(poppler is evince's backend)

  Changed 6 years ago by cscott

I think we should probably close this bug, 1286, and 4281 and 5535, and reopen new bugs with any new bloat found. I think I've already addressed most of the reasonable stuff in 4281 and 5535.

We should also re-audit http://dev.laptop.org/~rwh/announcer/joyride_vs_update1.html just to make sure we've got all the F9 bloat.

  Changed 6 years ago by cscott

  • status changed from new to closed
  • resolution set to fixed

Closing this bug.

Note: See TracTickets for help on using tickets.