terrible SD write performance

[dsd]

Our system feels slow and our syncs take ages because of terrible SD card write performance. I am testing on a system which is marked as having a sandisk C24G

To reproduce: boot a system, press ctrl+alt+f2, run "sync" at bash prompt. It will take 5-10 seconds.

blktrace shows that only ~100kb of data is being written in those 5-10 seconds - approximately 12kb/sec.

I traced this down to the sdhci level. The individual small, scattered writes that are being performed are often taking 400-600ms to complete (even just for a 2 sector write). I went as far to check that that delay definitely occurs at the point where the sdhci command has been dispatched and before the interrupt arrives. (there is no measurable delay in the lead-up to command submission, or the processing of the interrupt)

I will attach some logs showing the write access pattern and the delays, and I will write a C program to reproduce this slow set of writes and hopefully act as a test case.

[dsd]

logs from a 20 second sync

[dsd]

sdhci write logging kernel patch used to produce the above

[dsd]

C program that reproduces the same writes

[dsd]

log of the program running on the same system, same SD card

[dsd]

a 2nd invocation of the same prog on the same setup

[dsd]

same prog, same SD card, but inserted into my Dell laptop (also sdhci)

[dsd]

Dell setup, repeated

[dsd]

My simple test application seems to reproduce the issue. It would be interesting for other people to try it. Note that it will probably destroy your filesystem.

For some reason, in the test app some of the seeks fail and it doesn't try that particular write, but the other ones show the issue.

As you can see from the attachments above I also moved the SD card over to my main laptop and reproduced the issue. Perhaps this SD card just sucks...

There is a frightening correlation between fast and slow sectors on all of the above tests. For example, sector 2416648 always seems to be fast, and sector 2400280 always seems to be slow. I also noticed a frightening correlation between fast and slow sectors when I (separately) repeated the test on a 2nd SD card, this one from a different manufacturer (Transcend)... !?

[Quozl]

Triage.

[wad]

We should add your test program to the SD card qualification process along with the SD wear leveling testing.

I'm not happy with the performance of the Class 2 SD cards, but they generally perform better than JFFS2 on the XO-1, and without loading the processor.

This is probably improved on Class 6 cards, but there is strong pressure on the laptop price right now, and we can't afford the extra couple of dollars.

[anonymous]

Milestone 1.5-software deleted