Sugar should provide activities (at least python ones) with temp file facilities

[tomeu]

Desired features:

• each activity has a subdir in /tmp
• files are private to that process id
• autodelete on activity shutdown
• size limit
• perhaps delete the oldest file when that limit is reached?

[mstone]

This is a part of containerization. Of the items in this list, we presently have parts 1-3 working in our prototype security service, Rainbow.

Caveat 1: finalization needs to be more robust in the face of reboots, low-memory, etc.

Caveat 2: there are some sticky issues involving the interaction between filesystems, mount-points, hardlinks, and VServer security context (xid) file-tagging that we're presently working out. Rainbow and the Datastore are going to have to start cooperating soon (post Trial-2) in order to get this right.

Part 4 (size limits) is being put off for the moment while we work on implementing other capabilities.

Why do you think that deleting the oldest file is the right default?

See also #1992.

[tomeu]

Replying to mstone:

Why do you think that deleting the oldest file is the right default?

I don't think so, was just an example of criteria we could use for freeing space. Surely there are better ones.

[tomeu]

Not in roadmap.

[tomeu]

Don't have a pressing need for this right now. Please comment if anyone think otherwise.

[mstone]

Turns out that putting activity instances into a separate namespace from Sugar is a bad idea; people really seem to want to be able to communicate with sugar through their tmp-dir. Fortunately, this is one wish that we can safely grant.

Solution: leave activities in the same namespace as Sugar but be sure to mount the tmpfs you're going to give to the activity inside an appropriately permissioned gate-dir that sugar controls. This way Sugar and the activity share access to a RAM-backed space that no one else can get to, the size of the space is still restricted, and we remain agnostic about when the space will actually be reclaimed (which is important since the activity instance life-cycle is still not adequately pinned down).

[kimquirk]

moving to update2

[cscott]

I'm nominating this for Update.1, as otherwise activities can't use their temp space to write files to the journal/datastore. Both Pippy and Web want to do this in order to implement 'view source'. There is a workaround -- they write to the persistent 'instance' directory instead -- but if Michael can fix this by Friday I'd much rather see a proper fix go in.

[tomeu]

I don't understand how using tmp is the correct approach and using instance is a workaround.

Writing the data you want to put inside the DS to a file in instance is the recommended approach.

Why do Pippy and Browse want to write those files in temp? I think temp is for small and short-lived files.

[marco]

I think the "work around" is fine for Update.1

[mstone]

lumpy_ pointed out to me that I should be protecting /tmp from malicious writes by activities.