Datastore should check versions and recreate indexes automatically.

[jg]

Build 541 upgraded from build 540 via the upgrade path would not put up the USB icon in the journal.

Deleting the index files from the USB key "fixed" the problem; the key would then be recognized and the icon on the journal restored.

I think we need version checking of the datastore index files implemented so that these sorts of failures don't happen in the field; the old index should be removed (or potentially converted), and the index rebuilt.

[tomeu]

Ben, I thought your commit 21b3150356031bfed0e833d148c4f370364c3164 of Fri, 27 Jul 2007 fixed this.

[tomeu]

Mounting an usb device failed because of this:

DBusException: org.freedesktop.DBus.Python.xapian.DatabaseCorruptError: Traceback (most recent call last):
  File "/home/tomeu/sugar-jhbuild/build/lib/python2.5/site-packages/dbus/service.py", line 664, in _message_cb
    retval = candidate_method(self, *args, **keywords)
  File "/home/tomeu/sugar-jhbuild/build/lib/python2.5/site-packages/olpc/datastore/datastore.py", line 76, in mount
    mp = self.connect_backingstore(uri, **_options)
  File "/home/tomeu/sugar-jhbuild/build/lib/python2.5/site-packages/olpc/datastore/datastore.py", line 168, in connect_backingstore
    bs.initialize_and_load()
  File "/home/tomeu/sugar-jhbuild/build/lib/python2.5/site-packages/olpc/datastore/backingstore.py", line 74, in initialize_and_load
    self.load()
  File "/home/tomeu/sugar-jhbuild/build/lib/python2.5/site-packages/olpc/datastore/backingstore.py", line 456, in load
    super(InplaceFileBackingStore, self).load()
  File "/home/tomeu/sugar-jhbuild/build/lib/python2.5/site-packages/olpc/datastore/backingstore.py", line 244, in load
    im.connect(index_name)
  File "/home/tomeu/sugar-jhbuild/build/lib/python2.5/site-packages/olpc/datastore/xapianindex.py", line 84, in connect
    self.write_index = secore.IndexerConnection(repo)
  File "/home/tomeu/sugar-jhbuild/build/lib/python2.5/site-packages/secore/indexerconnection.py", line 46, in __init__
    self._index = _xapian.WritableDatabase(indexpath, _xapian.DB_CREATE_OR_OPEN)
  File "/home/tomeu/sugar-jhbuild/build/lib/python2.5/site-packages/xapian.py", line 2759, in __init__
    _xapian.WritableDatabase_swiginit(self,_xapian.new_WritableDatabase(*args))
DatabaseCorruptError: Cannot open tables at consistent revisions

[marco]

Tomeu, is this actually a blocker? Do you think you could come up with a patch since Ben is in vacation?

[tomeu]

After I come up with a fix for #3180, usb devices will be reindexed and mounted after a failure.

I'm more worried about the flash index, but I think the datastore format didn't changed between trial2 and trial3.

[tomeu]

A fix for #3180 has been pushed for Trial-3. For version mismatches, we will recreate the index (but loose all existing metadata).

I think we didn't change the index format for Trial-3, but would be good that someone tests the upgrade process from Trial2 to Trial3 and check we don't loose data in the process (#3389).

[kimquirk]

For FRS, can the datastore provide information if the USB is corrupted. Maybe the indexes should be more transactional.

[marco]

The specific bug is fixed.