Ticket #8190 (new defect)

Opened 6 years ago

Last modified 5 years ago

olpc-update's irsync_pristine step always fails with "Contents manifest failure at line 383" on localtime

Reported by: skierpage Owned by: cscott
Priority: normal Milestone: 9.1.0-cancelled
Component: upgrade utility Version: Development build as of this date
Keywords: cjbfor9.1.0 Cc:
Action Needed: test in release Verified: no
Deployments affected: Blocked By:
Blocking:

Description

As reported in bug 5805 comment #5 and in bug 5954, in olpc-update, irsync_pristine fails always/a lot, on the file "localtime":

Trying irsync_pristine update from /tmp/tmpdKozIo
- Cleaning tree.
- Fetching contents.
- Performing incremental rsync.
... [many minutes go by]
Verifying update
Contents manifest failure at line 383
Last file examined: localtime

I've seen this on three out of 4 olpc-updates, latest is joyride-2301 to 8.2-757. I have /etc/localtime and /etc/avahi/etc/localtime that are dated 2007-11-02.

If the step is always going to fail, why bother doing it? Seems like this might be worth investigating.

Attachments

tmpTEslAQ.txt (2.1 kB) - added by thomaswamm 6 years ago.
Terminal capture of $ sudo olpc-update 8.2-757 (from joyride-2301). Looks not too smooth. But took only 20 minutes after midnight PDT.
tmpdrUuaJ.txt (1.4 kB) - added by thomaswamm 6 years ago.
Terminal transcript of $ sudo olpc-update 8.2-759 (from 8.2-757). Took under 30 minutes near midnight. Still has localtime bug, but went smoother than previous update to 8.2-757.
update760.txt (1.8 kB) - added by thomaswamm 6 years ago.
golden Terminal transcript for $ sudo olpc-update 8.2-760 (it went very well).

Change History

follow-up: ↓ 3   Changed 6 years ago by mavrothal

Same here with a unsecure G1G1 machine running joyride 2298/q2e14
and then it attempts

irsync_dirty

and in the middle of it (after ~5 minuts) crashes gives a white screen then a black screen with running text (that I can not read) and restarts Sugar (it does not reboot) with my old build

follow-up: ↓ 5   Changed 6 years ago by thomaswamm

Similar here, on olpc-update to jr2301 and before that to jr2263.

olpc-update takes so much time that I became a fan of clean-installs from USB stick. Then I can quickly reflash anytime I want.

in reply to: ↑ 1 ; follow-ups: ↓ 4 ↓ 6   Changed 6 years ago by skierpage

Replying to mavrothal:

and in the middle of it (after ~5 minuts) crashes gives a white screen then a black screen with running text (that I can not read) and restarts Sugar (it does not reboot) with my old build

That sounds like bug #5954. As I understand it, olpc-update uses a lot of memory and the kernel's out-of-memory handler kills the X Window System which restarts. In a terminal enter dmesg | less and you may see references to memory and oom.

As for this bug, m_stone commented on IRC "irsync_pristine is always going to fail until we actually manage to make the root filesystem readonly. (we were close to that for a while, then were pulled farther away, and now are getting close again)" Maybe that option should be removed from "hints" in /usr/bin/olpc-update.

in reply to: ↑ 3   Changed 6 years ago by mavrothal

Replying to skierpage:

That sounds like bug #5954. As I understand it, olpc-update uses a lot of memory and the kernel's out-of-memory handler kills the X Window System which restarts. In a terminal enter dmesg | less and you may see references to memory and oom.

So what's the remidy? The #5954 ticket has at the end a link to an addition to the "post-olpc-update" script http://dev.laptop.org/ticket/28#comment:19 on ticket #28, but is totaly unlear (to me) "which, where and when" we are takling about.

in reply to: ↑ 2   Changed 6 years ago by cscott

Replying to thomaswamm:

Similar here, on olpc-update to jr2301 and before that to jr2263. olpc-update takes so much time that I became a fan of clean-installs from USB stick. Then I can quickly reflash anytime I want.

You probably want to use the '-f' option to olpc-update, then. This is not enabled by default because it is hard on our poor rsync server.

in reply to: ↑ 3 ; follow-up: ↓ 8   Changed 6 years ago by cscott

Replying to skierpage:

As for this bug, m_stone commented on IRC "irsync_pristine is always going to fail until we actually manage to make the root filesystem readonly. (we were close to that for a while, then were pulled farther away, and now are getting close again)" Maybe that option should be removed from "hints" in /usr/bin/olpc-update.

That's not exactly true; irsync_pristine has a number of hacks hard-coded to make it expected to succeed on 'normal' systems. But it's not perfect (it will be in the 9.1 timeframe, as mstone says), and you can still dirty your pristine copy in some situtations.

This seems to be one of them: how on earth are you writing to /etc/localtime? Sugar's timezone setting feature explicitly does *not* set this. Elsewhere /etc/avahi/etc/localtime was fingered as a co-culprit -- is there some network environment in which avahi can be induced into writing to /etc/localtime? That certainly doesn't happen at 1cc. I'd be interested in more information. Perhaps an Apple machine is on the network, broadcasting "interesting" mDNS services?

  Changed 6 years ago by mikus

Seen the "manifest failure, Last file examined: localtime" situation many many times.

I notmally use 'olpc-update --usb', but nowadays, because of this exact problem, I include the --full parameter when I use olpc-update..

in reply to: ↑ 6   Changed 6 years ago by mavrothal

Replying to cscott:

Perhaps an Apple machine is on the network, broadcasting "interesting" mDNS services?

Good call!. An apple machine is on the same WiFi network as a client. However, I have no idea what is broadcasting, if anything. How do I find out?

  Changed 6 years ago by kimquirk

  • milestone changed from Not Triaged to 8.2.1

  Changed 6 years ago by ridderman

I see this on all updates, and I don't have any Apple devices on my network. I'm also on a G1G1 laptop. We'll see if the fix that went into joyride works for me.

in reply to: ↑ description   Changed 6 years ago by akonstam

Replying to skierpage:

As reported in bug 5805 comment #5 and in bug 5954, in olpc-update, irsync_pristine fails always/a lot, on the file "localtime": {{{ Trying irsync_pristine update from /tmp/tmpdKozIo - Cleaning tree. - Fetching contents. - Performing incremental rsync. ... [many minutes go by] Verifying update Contents manifest failure at line 383 Last file examined: localtime }}} I've seen this on three out of 4 olpc-updates, latest is joyride-2301 to 8.2-757. I have /etc/localtime and /etc/avahi/etc/localtime that are dated 2007-11-02. If the step is always going to fail, why bother doing it? Seems like this might be worth investigating.

I had exzactly the same error in installing v 711 and v 757 today. The installation seems to suceed so I an unclear what exactly the implication of the error.

Changed 6 years ago by thomaswamm

Terminal capture of $ sudo olpc-update 8.2-757 (from joyride-2301). Looks not too smooth. But took only 20 minutes after midnight PDT.

  Changed 6 years ago by cscott

There were fixes to the MemoryError and dirty /etc/localtime problems landed in joyride-2369; please help test.

  Changed 6 years ago by cscott

  • next_action changed from never set to test in release

olpc-update 2.17-1

Added to stable stream; should be in 758 and following.

  Changed 6 years ago by ridderman

I updated from joyride-2373 to joyride-2386 and did not see this bug. The irsync_pristine worked for me for the first time in ages. Nice work!

Changed 6 years ago by thomaswamm

Terminal transcript of $ sudo olpc-update 8.2-759 (from 8.2-757). Took under 30 minutes near midnight. Still has localtime bug, but went smoother than previous update to 8.2-757.

  Changed 6 years ago by cscott

irsync_pristine worked for me from joyride-2414 to 2420 or so.

I finally found the likely culprit overwriting /etc/localtime. When we make our DHCP request, we explicitly ask for "Subnet-Mask, BR, Time-Zone, Default-Gateway, Domain-Name, Domain-Name-Server, Hostname, YD, YS, NTP" using DHCP option 55 (section 9.8 at http://tools.ietf.org/html/rfc2132). I'm guessing dh_client is obliging by overwriting our /etc/localtime for us based on the Time-Zone.

Is this a bug or a feature, I wonder?

  Changed 6 years ago by thomaswamm

From 8.2-759 I did $ sudo olpc-update 8.2-760 and had my smoothest-ever update experience, pristine, with no localtime nuisance. It took 17 minutes to "update succeeded".

I will attach a Terminal transcript just for the record, in case anyone has a bad experience and wants to compare notes.

Changed 6 years ago by thomaswamm

golden Terminal transcript for $ sudo olpc-update 8.2-760 (it went very well).

  Changed 5 years ago by mstone-xmlrpc

  • keywords cjbfor9.1.0 added
  • milestone changed from 8.2.1 to 9.1.0

Pushing out to 9.1.0, per edmcnierney's request.

Note: See TracTickets for help on using tickets.