[owncloud-devel] Experience with WD Chalenge like setup

Jos Poortvliet jospoortvliet at gmail.com
Thu Dec 17 21:36:11 GMT 2015

On Thursday 17 December 2015 22:07:34 Michal Hrusecky wrote:
> Hi,
> while discussion is going on, I have a few questions regarding everyones
> experience with running similar setup to what is proposed in WD
> Challenge. I have at home BananaPi with external USB harddrive running
> ownCloud now and while installing it I did few decisions that might be
> relevant to the challenge as well, so I was wondering whether you have
> similar experience/setup or something else (will help us shape the image
> and improve our home setups).
> Decisions I made:
> -----------------
> 1) Database on SD card
> I have whole system and even the ownCloud database on SD card. Reasoning
> behind it is that even though it is slow, my externel harddrive is put
> asleep most of the time and spinning it up would take quite some time.
> So I prefer to get immediately something responding slowly than
> something that needs to wake up and than performs. And I don't want to
> have it spinning all the time as that is the point of going for ARM
> board - saving up some energy.

Yeah, but it will kill the SD card in a few weeks ;-)

These SD Cards SUCK at doing tiny writes - they have huge blocks (think 4MB or so) so every time you write, like 1KB of data, it writes 4 MB. Slow AND it will exhaust the number of writes the card can sustain in no time. SD cards are made for photo's, not databases :(

> 2) SQLite instead of MySQL
> SQLite has plenty of issues, but I ended up using it instead of MySQL as
> I have a feeling that on SD card it is faster. But didn't run any
> benchmark and didn't tuned up MySQL much. On real drive, MySQL should
> beat SQLite in performance without any problem, but I had the feeling
> that writes/reads were much more scattered all around and thus slower
> than SQLite when running on slow weird storage like SD card. Did anybody
> did some benchmark? What is your experience? I actually migrated from
> SQLite to MySQL and than back, but maybe I should have spend more time
> tunning MySQL.

Yeah, SQLite is cool - but not very good with ownCloud, really. We can try, yes, but then we might need to do some patching in ownCloud to make it more efficient with SQLite.

That's the same as with NGNIX, btw, it works but there are rough areas so we might need to patch ownCloud. Then again, that is not a big problem, I have heard that ownCloud is open source so we can probably get those patches upstream :P

So - don't listen too much to grumpy me, give it a try. But then a serious try: put the 1TB drive full of pictures and documents, then create 10 users, let 5 run the sync client and let 2 others view pictures.

If that works fast, you're ready for release! ;-)

> My performance boosting TODO:
> -----------------------------
> 1) Copy database to tmpfs on start and sync it back regularly to keep
> persistence, gain speed but end up with possibility of loosing last X
> minutes of history. Would require quite some scripting which I didn't
> get to yet, but wondering whether somebody does have something like that
> already and what people think about it :-)

THAT is a great trick, yes, and that might work very well. Of course we need to think about losing data but if only the DB is updated and not the file system, it is probably fine.

> 2) Use squashfs + overlayfs for OS. Found out that reads from SD card
> are quite slow and as I don't want to have my HDD spinning all the time,
> sounds like some of the reads from OS can be speed up by using squashfs
> and thus compressed filesystem. Would require special mounting on start
> but could safe both space and time. Not for data, but should allow to
> fit whole OS on small SD, maybe even multiple times and could improve
> performance over direct SD card. Did some basic unfair testing and
> looked like reading files from squashfs on top of SD card is faster than
> reading from SD card directly.

Our idea was to use the SD card exclusively as a boot loader, do everything from the hard drive... Because of the damage I talked about earlier.

> So, what is your experience? What do you think about my decisions and
> ideas?

Keep it coming!

> _______________________________________________
> Devel mailing list
> Devel at owncloud.org
> http://mailman.owncloud.org/mailman/listinfo/devel

Everything I do and say is based on my view of the world today. I am not responsible for changes in the world, nor my view on it. Everything I say is meant in a positive and friendly way, unless explicitly stated otherwise.
find me on blog.jospoortvliet.com

More information about the Devel mailing list