some files shouldn't be compressed by jffs2

[tomeu]

Storing a PDF of 35MB takes 25s. to be written to NAND, processor bound.

If this is caused by JFFS2's on-the-fly compression, marking those files as already compressed using chattr or xattr would make these operations much faster.

Also, I presume reading perf would also be improved.

[pascal]

Maybe jffs2 should detect it all by it self.

It is trivial to detect that a block didn't compress well, and then assume that the next equally sized block will not compress either and the just store the data. This will result in testing in the order of sqrt(filesize) bytes (is my math right?) to see if the file will compress, or 187KB in tests for the 35MB file. Increase the increment to 4 times and you need to compress something like 13KB to test the entire file.

Perhaps you can use a fixed maximum block size after initially determining that it will not compress, so the detection of compressible data later in the stream does not suffer so badly.

[tomeu]

Jim, can we up the priority of this one?

Taking all the cpu during downloads because of compression (#5235) when most files are already compressed is no good. It slows all other operations in the laptop, and I think we cannot expect the user to wait for the download to finish before doing other tasks.

Pascal's suggestion would be much better than having to mark the files as uncompressed based on their mime type.

[ixo]

How about auto-detection of files already compressed? Aka PNG , JPG, gz, zip, etc... is an easy test of the first 10 bytes, help terminal the 'likely format of the file?

[cjb]

I have an activity that serves content out of a large, heavily-compressed .bz2 on demand. It would be much faster if I could disable jffs2 compression for those nodes.

[dsaxena]

Replying to ixo:

How about auto-detection of files already compressed? Aka PNG , JPG, gz, zip, etc... is an easy test of the first 10 bytes, help terminal the 'likely format of the file?

Ick, no. Probably the best way to do this is via an xattr and/or possibly a new generic O_NOCOMPRESSION flag that can be used by any FS since compression is available on various file systems these days.

There are some patches out there in the embedded world for something along these lines (though not on JFFS2). I'll poke some folks about this.

[cjb]

There are some patches out there in the embedded world for something along these lines (though not on JFFS2). I'll poke some folks about this.

Thanks. I'm sure we'd more than double the speed of the wikipedia activity by being able to stop zlib-uncompressing the already-heavily-compressed data -- so this is useful even if it's not automatic to start with.