Opened 7 years ago

Closed 7 years ago

#5637 closed defect (fixed)

orphaned files

Reported by: walter Owned by: mstone
Priority: blocker Milestone: Update.1
Component: security Version:
Keywords: rainbow-integration Cc: ivan, jg, bernie, mstone, cscott
Blocked By: Blocking:
Deployments affected: Action Needed:
Verified: no

Change History (15)

comment:1 Changed 7 years ago by jg

  • Cc jg added
  • Milestone changed from Never Assigned to Update.1
  • Priority changed from high to blocker

This seems pretty serious to me. How hard is it to fix?

comment:2 Changed 7 years ago by tomeu

If I correctly understand the issue, this is not a problem at all after ship.2. We stopped copying files to there and are hard linking them instead.

Could someone try to reproduce the issue in one of the update.1 builds?

comment:3 Changed 7 years ago by tomeu

  • Cc bernie added

But I guess we should anyway clean up that dir after booting, right?

comment:4 Changed 7 years ago by bernie

What would be a reliable way to identify the orphaned files? We'd better be extra cautious when deleting user data.

comment:5 Changed 7 years ago by tomeu

We don't need to delete any file. We wish to remove the hard links from that dir, because I guess every hard link takes up some space.

comment:6 Changed 7 years ago by jg

The links take almost no space.

If there is a hard link to a file, even if the other link is removed, the space will not be freed...

So long as the link count is one or higher, files will not actually be deallocated.

comment:7 Changed 7 years ago by tomeu

  • Cc mstone cscott added

I cannot see this happening any more in update.1 builds, although I guess some hard links could be leaked in /activities/uid_to_instance_dir/* by bad-behaved activities. I'm not sure if those directories are cleaned up when booting.

Perhaps Ivan has some plans for the new datastore that won't use hard links?

Anyway, I don't see this as a blocker for update1.

comment:8 Changed 7 years ago by mstone

  • Keywords rainbow-integration added

comment:9 Changed 7 years ago by jg

tomeu, mstone: if this has been tested as fixed, then please close the bug...

comment:10 Changed 7 years ago by marco

  • Component changed from journal-activity to security
  • Owner changed from tomeu to mstone

jg, not fully fixed yet.

Michael is going to see if we can have rainbow rm -rf the instance dir, as a solution for Update.1. (on the longer term we will have to revisit the exact activity instance lifecycle)

comment:11 Changed 7 years ago by chihyu

I am creating test cases for this issue, but the original bug description has disappeared from the wiki. It will be great if someone could suggest how to test this issue, and whether it should be tested in Update.1.

Thanks a lot!

Test cases:

comment:12 Changed 7 years ago by gnu

The original problem, I think, is that the Datastore made copies of files that were deleted or opened from USB storage devices or SD cards. This was popular, as G1G1 people tried to play movies or music from USB disks or flash sticks. Copying in a 300MB movie, however, tended to fill the NAND filesystem. And then nothing deleted those copies of the files. These are the bugs: #5719, #5744.
Here is another bug that mentions this issue, in the context of ship.1 to update.1 upgrades: #5955.

Recipe to reproduce:

  • Install Ship.1 build 650.
  • Plug in a USB stick containing a large file (e.g. the 128MB Ogg Theora from #5744)
  • Attempt to open that file (resulting in a copy in NAND).
  • Let the USB light stop blinking (Sugar is done copying).
  • Check "df" to see how much space is occupied, and check "du /home/olpc/.sugar/default/data/"
  • Eject (in journal) and physically remove the USB stick
  • Bring up a terminal, and upgrade to update.1.
  • Reboot into update.1
  • Check "df" to see how much space is occupied, and check "du /home/olpc/.sugar/default/data/"

to make sure that all the files are gone from there.

comment:13 Changed 7 years ago by mstone

The rainbow side of this bug has been complete since rainbow-0.7.7.

comment:15 Changed 7 years ago by mstone

  • Resolution set to fixed
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.