[owncloud-devel] OwnCloud on WD Proposal

Stefan Bruens stefan.bruens at rwth-aachen.de
Sun Dec 20 19:16:28 GMT 2015


On Sunday 20 December 2015 17:49:37 Jos Poortvliet wrote:
> You shouldn't run ownCloud on a SD card anyhow - these cards are heavily
> optimized for sequential read and write performance, sporting block sizes in
> the megabytes. Storing a database on that with its random write patterns
> reads to massive write amplification: for every tiny write the SD card
> writes a full block of several MB's, burning through the lifetime write
> capacity of the card in months, weeks or even days if you use it heavily.

Jos, can you please stop these unwarranted claims.

While it is true that a small write may lead to a much larger write, this is 
in general not the case. NAND flashes today have pages of 2kByte to 8kByte, 
thats the smallest amount you can write. Everytime you "overwrite" a logical 
block of data, it is just marked as stale data.

The larger erase blocks only comes into play on garbage collection. Garbage 
collection happens if
- there are no erase blocks with unused (neither used nor stale) pages, or
- occasionaly, during idle periods
Garbage collection happens more frequently if the media is filled up.

Although SD cards *are* actually much slower on random access than on 
sequential access, these are not worse than HDDs. HDDs have access times of
~10ms, so you have ~100 Ops/s. SD write access is about the same, random read 
access is about one order of magnitude faster.

The RPi is also limited by its small memory. As soon as you have to move dirty
memory pages to storage or drop data from caches, the USB disk is a 
bottleneck.

For the matter of write amplification, you should differentiate between write 
amplification caused by:
(1) the database
(2) the file system
(3) the flash translation layer (i.e. inside the storage media)

In fact (1) can alone lead to a write amplification by a factor of two. SQLite 
in its default journal mode writes everything to the journal *and* to the 
database file. SQLite since 3.7.0 knows a Write-Ahead-Log (WAL) mode where 
changes from the journal are only written back to the database during 
checkpointing, coalescing many small writes into larger ones. Other databases 
use similar schemes to SQLites WAL.

(2) may be problematic some filesystems not used properly. For e.g. BTRFS you 
should make sure COW is disabled for database files.

(3) may be an issue if you have old, bad, cheap SD cards. You can not rule 
this out in general, but if the card has a somewhat decent random write 
performance (~1MByte/s), the card needs to have a proper FTL.

IMHO is is a very good idea to put both the OS and the database on SD card. 
Even a small (8GByte) SD will be largely unused, which leaves plenty of space 
for flash wear leveling end ensures seldom run of the garbage collector.

Kind regards,

Stefan

-- 
Stefan Brüns  /  Bergstraße 21  /  52062 Aachen
home: +49 241 53809034     mobile: +49 151 50412019
work: +49 2405 49936-424


More information about the Devel mailing list