Autoinstallation of RPMs

[cscott]

Developers have a peculiar use case: they often want to install multiple additional packages on top of the base build, and they are willing to do maintenance to fix things that break.

A proposed mechanism is to have a signed script on an attached USB or SD device which is run by olpc-configure on reconfigurations (first boot of a new OS build). The script may be signed by the public/private keypair of the XO to tie it to a specific machine, minimizing use of this vector for trojans. (Reflashes nuke the keypair; an alternative is to simply incorporate a hash of the SN and (hidden) UUID to equivalently tie the script to a specific machine.)

Ultimately, the desired use case is something like the following:

# olpc-install emacs
# olpc-sign-cache

This hypothetically would use yum and the network to download emacs and its dependent RPMs and store them on an appropriate USB/SD device. The olpc-sign-cache command would create an appropriate script to install these RPMs, 'sign' it to tie it to the current machine, and install it under the appropriate filename on the USB/SD device.

First step, however, is just to provide the basic mechanism; the friendly tools can come later.

To think about: in addition to an attached USB or SD device, we could also consider looking in /home/olpc/.foobar-cache, which may be appropriate for 'small' customizations.

This mechanism is dangerous: countries should be discouraged from using this in school deployments because updates may break kids' laptops in arbitrary ways.

[dgilmore]

yum has a mechanisim that allows you to script installing packages

for more info see http://skvidal.wordpress.com/2008/01/18/yum-on-the-xo/

so we could put a hook in olpc-update that would install based on a script in users home dir. we could preserve the yum cache and need to download very little.

[cscott]

The hook goes in olpc-configure, not in olpc-update, but a yum hook was exactly the sort of thing we were thinking of.

[mtd]

I don't understand where olpc-configure is (you'll tell me in a second), but I guess this isn't that great because it's pre-potential-reboot, right?

http://pastebin.be/8998

[cscott]

Yes. As I wrote above, the correct time to do this is on the first boot of the new system, *not* in olpc-update. olpc-configure handles first boot configuration. It is in the olpc-utils package.

[mtd]

Thanks for the pointer (sorry was late and I didn't have the energy even for one last find command). I'll have a look at olpc-configure. One thing that did occur to me is that I'm not sure what you're envisioning you'd like catered for most / at all out of these (not comprehensive) use cases, but the reason my patch came to mind was:

1) I know what packages I want installed after a new joyride blows them away again, and that package set doesn't change very often...so right after I olpc-update, I just run two scripts, one which patches various files (and yum installs patch first), and another which just has a long "yum -y install <mystuff>" set of commands (I could use yum shell as seth suggests but I don't see the advantage for my case);

2) I have no need to keep the packages off-network - if I'm olpc-update-ing, I've got the network; and

3) I trust the person running olpc-update, so I can trust script(s) that run right after it. And I don't run the scripts unless I'm updating my XO. Thus I don't need the signing part of your proposal (though it's cool); and

4) as a corollary of 2) + 3), I don't need a (signed) cache, though I could easily use my sd card as (an unsigned) one with yum localinstall instead of yum install in my post-olpc-update script.

I know you stated your target use case, but you/others may want to consider some/all of these different, but still developer-ish, use cases:

1) one knows what packages one wants to install, and one runs olpc-update from the network, so it's very tempting to install the packages when you know you have the network, right at the end of olpc-update (as my patch did; it's probably not surprising that this is what I think I need :), personally);

2) same as 1) except install is non-network and network isn't available on first boot (do you care about this? Would it be an /etc/NetworkManager/dispatcher.d/olpc-update-ifup type of problem, or is there a less-frequently-invoked alternative?);

3) same as 1) but one needs the network for packages but is olpc-updating from non-network, and the network isn't available now nor will be at first-boot (is this too pathological a case?).

I realize I'm totally hijacking your trac ticket and you probably aren't excited about that, so I'll stop now :).

[cscott]

In mail to devel@ I've proposed limiting this mechanism to machines with dev keys installed, in order to manage the deployment risk I mentioned in the bug summary above.

Michael has implemented a first draft of this mechanism as a patch to olpc-configure. The draft does not consult external devices, and does not check a dev key:

http://lists.laptop.org/pipermail/devel/2008-March/011554.html

Benjamin M. Schwartz had further comments about Bitfrost P_SF_RUN incompatibilities:

http://lists.laptop.org/pipermail/devel/2008-March/011583.html

which I believe would also be addressed by the dev key limitation, if one assumes that a dev key effectively turns off Bitfrost. Other mechanisms to address the specific needs of developers may be more appropriate.

[mstone]

In subsequent discussions, we have also realized that we would be willing to install RPMs signed by a trusted key at any time and any RPMs at all _only on first boot_ into a new OS tree.

[bemasc]

Replying to mstone

If the boot counter is kept in the firmware, and preserved over firmware upgrades, then this seems like a secure option, provided that activation is required first. It seems very fragile though. What if I accidentally reboot a machine before completing the unsigned RPM installation?

The underlying statement here is: we will allow the deployers to make any modifications they choose to the software on the machine. If that's the case, why not just delegate signing authority so that deployers can create new custom images and RPMs for their machines at any time?

[cscott]

Replying to bemasc:

Replying to mstone If the boot counter is kept in the firmware, and preserved over firmware upgrades

It is not, and cannot be for Gen 1.

The underlying statement here is: we will allow the deployers to make any modifications they choose to the software on the machine. If that's the case, why not just delegate signing authority so that deployers can create new custom images and RPMs for their machines at any time?

http://wiki.laptop.org/go/Firmware_Key_and_Signature_Formats#Version_2

This has gone afield, though: the original use case is for *developers* to make it easy to follow new releases (from OLPC or Uruguay or whomever) yet maintain a set of installed "extra" RPMs.

Michael's patch, with a simple dev key restriction added, seems the most direct means to address this use case at the present time.

[cscott]

I should mention that the bitfrost.leases.crypto package, already installed on XOs as part of the olpcupdate package, makes it pretty easy to check a dev key. I'll add a 'verify_dev' method to it to make it even more straightforward.

http://dev.laptop.org/~cscott/joyride-2083-api/bitfrost.leases.crypto-module.html

[mikus]

The original description of this ticket presents two goals in addition to "automation": [1] preserving any additional packages when a base build gets replaced (e.g., that's 'olpc-install'); [2] providing that a system update after the base install is "secure" (e.g., that's 'olpc-sign-cache').

A "non-automated" goal [1] is already available to experienced (with 'root' privilege) users - through 'yumdownloader --resolve'. [Note that different people will want different additional packages.] Afterwards, 'yum localinstall *.rpm' can be used -- is it worth modifying something like 'olpc-configure' to avoid having to manually issue that command ?

---

I'm not sure that goal [2] is needed in a development environment -- currently it is far too easy to bypass security by getting 'root' privilege (using 'su'; using the 'root' icon in Terminal; using a text-console <ctl-alt-F1> logon).

But for secure machines (i.e., for non-Developers), a facility is needed that allows secure updating of system packages after install. [Similar to how the 'customization key' facility could be used to update Activities.] Such an "update securely" facility *would* need signing the input.

[cscott]

Replying to cscott:

In mail to devel@ I've proposed limiting this mechanism to machines with dev keys installed, in order to manage the deployment risk I mentioned in the bug summary above. Michael has implemented a first draft of this mechanism as a patch to olpc-configure. The draft does not consult external devices, and does not check a dev key: http://lists.laptop.org/pipermail/devel/2008-March/011554.html

I'm attaching a slight variant of Michael's patch, which does check dev keys. It doesn't check external devices, since udev/sugar haven't mounted any at this point in the initscripts. =(

As mikus has pointed out, 'yumdownloader --resolve' might be used to create this cache. What does 'yum localinstall *.rpm' buy you over 'yum -yt --nogpgcheck install $pkgs'?

The original proposal had a signing step to prevent a cache on removable media becoming an easy/sneaky trojan mechanism. Restricting to 'at first boot', dev keys, and the onboard NAND seems to be enough for the moment: all we can do it make it more difficult to install a trojan by this means than it is to switch to vt 1 and type 'rpm -Uvh /media/*/*.rpm', as mikus correctly notes.

Comments? Thoughts?

[cscott]

Proposed usb-customization patch, based on mstone's work.

[cscott]

The current patch looks for RPMs under /home/olpc/.custom/rpms -- anyone want to write a little olpc-install script (as per the original description) which uses yumdownloader --resolve to create a subdirectory under ~/.custom/rpms for this installation, downloads the requirements, and then installs them (presumably backing out the new dir if the installation fails)?

olpc-sign-cache can then just be a wrapper around olpc-contents-create and olpc-contents-verify with some signature magic thrown in from bitfrost.leases.crypto.

[cscott]

What is 'yum localinstall' actually doing for us, that 'rpm -Uvh *.rpm' isn't? I ask because 'yum localinstall' is a minute or so slower than the equivalent rpm command.

[cscott]

Comparison:

  $ yumdownloader --resolve emacs
  $ time sudo rpm -Uvh *.rpm
  real: 1m43.6s
  -- erase packages, try again --
  $ time sudo yum -yt --nogpgcheck localinstall *.rpm
  real: 3m36.2s

Ouch!

[cscott]

What is 'yum localinstall' actually doing for us, that 'rpm -Uvh *.rpm' isn't?

From mail from mikus:

Presumably "yum' records what it did in its own log, whereas 'rpm' by itself does not affect the yum log. Also, 'yum' (depending on how it is set up?) saves what it installs - one needs 'yum clean' to empty the yum cache; 'rpm' (as far as I know) does not cache the data it works with.

I'm the one who suggested 'yumdownloader --resolve'. With packages which I expect to continue to be available in (fedora?) repositories I will use vanilla 'yum' - it downloads the "rpms" and installs them. With packages that I had to go find, I'll usually want to manually keep a copy on my "permanent" SD card.

I was faced with the problem that 'yum' did not give me "no-brains" access to the "rpms" it fetched. Thus to me discovering 'yumdownloader' was a godsend -- if I want to store a local copy of some "rpm" from an appropriate repository, I access that repository with 'yumdownloader --resolve' instead of with 'yum'.

Once the "rpms" have been downloaded, the question is how to organize them, in case the developer does not wish to (re)apply all the "rpms" he has stored. A common Linux way to organize materials is to use subdirectories. By saying to install packages from '*.rpm' I implied that the person doing the install was accessing a specific subdirectory (of stored "rpms").

For myself, I always use 'rpm -Uvh' to install locally stored packages, instead of 'yum localinstall'. (I don't need whatever additional services 'yum' provides.) But since all previous text in this ticket had talked only about installing with 'yum', I wrote up using 'yum localinstall' -- I saw no need to "break new ground" by suggesting *not* using 'yum' to perform the not-using-net install.

cscott says:

Is there any chance that installing the packages directly using rpm will confuse 'yumdownloader --resolve' later? If not, I'd say let's install using rpm. I'm not sure developers will quickly embrace a solution which adds almost 4 minutes to their upgrade process.

[bernie]

Is there any chance that installing the packages directly using rpm will confuse 'yumdownloader --resolve' later? If not, I'd say let's install using rpm. I'm not sure developers will quickly embrace a solution which adds almost 4 minutes to their upgrade process.

Nope, yum does not store any additional information about the installed packages besides what's in the rpmdb.

Installing packages directly with rpm does not interfere with future yum operation.

[dgilmore]

the difference between "rpm -Uvh" and "yum localinstall" is that the yum command resolves all deps. so if you happened to be missing a rpm you needed it would download and get it also. rpm doesnt do that. so extra overhead is expected. using rpm you need to ensure all deps are present and accounted for.

[cscott]

Landed in joyride-2106 and documented at http://wiki.laptop.org/go/Yum.

Code can be reviewed at http://dev.laptop.org/git?p=users/cscott/olpc-utils-tmp;a=commitdiff;h=381cb7d55a4305d9e817b97dd1dfa727ce8a60bf, but note the change made in http://dev.laptop.org/git?p=users/cscott/olpc-utils-tmp;a=commitdiff;h=7ccbe298b23d8532d330521a9134cd01d0252250

Please test.

[gregorio]

Milestone Never Assigned deleted

[sj]

What's the status of this? I see value here for communities of non-developers in the field who want to maintain a set of rpm's for the builds they support, without using the big hammer of a developers key (which sacrifices other valuable features).

There was a support-gang discussion tonight about whether a dev key is needed to install rpm's. I would like to see dev keys return to the role of "lets you totally modify your OS and firmware" and not make that a barrier to rpm installs...

Perhaps we can add a sugar-control-panel option that says "allow rpm installation by customization key" (and perhaps another for "allow bundle updates by customization key", which might default to 'yes' -- something rather invasive, since I can give you a usb key to use and without your knowledge overwrite activities on your machine).

Assigned temporarily to 8.2.1 as this may be valuable for peru's summer build.

[mstone-xmlrpc]

Pushing out to 9.1.0, per edmcnierney's request.

[martin.langhoff]

The patch discussed here did make it into olpc-utils in 8.2.1. Some higher wishes didn't.