[owncloud-devel] WD Challenge Proposal - archlinux/btrfs/docker/avahi/rsync

Petros Angelatos petrosagg at resin.io
Tue Dec 15 08:29:15 GMT 2015


Hi all,

This is a great initiative, I really like it. Having an easy way of
setting up something like this is key for adoption. I'm writing this
proposal because it coincided with my intentions of building an
OwnCloud setup on a Raspberry Pi so it seemed a good idea to share my
design.

A few words about myself. My name is Petros. I am a founder and CTO of
Resin.io, a platform that allows you to deploy and manage containers
on embedded devices. We've done work to support multiple SBCs and even
more beefy machines. Some of them are the Raspberry Pi {Zero,1,2},
ODROIDs, Intel Edison, and Intel NUCs. I have worked extensively with
linux, from embedded devices to servers, have experience in
networking, filsystems, systemd, Yocto linux, kernel configuration
etc. My language of choice is nodejs with which I have developed a
number of backend services. I'm also interested in security, privacy,
and keeping control of one's data, which is my main motivation behind
this project.

Overview
========

The design consists of two parts.

1. A very minimal host OS, whose job is to boot up the board, mount
partitions, start network services, setup clock etc
2. A Docker container containing the OwnCloud application

Having the application in a Docker container means we can package all
the dependencies in it and be sure that it will run correctly. It also
makes updates a breeze since it's a matter of `docker pull` and
`docker run`. Finally, the OwnCloud Raspberry Pi image will not be
specific to this project. This means that someone with a Raspberry Pi
running Docker could easily add it without needed the host OS proposed
here.

All decisions bellow are made with the performance profile of the
Raspberry Pi 2 in mind.

Operating System
----------------

The obvious choice for the Raspberry Pi would be the raspbian
operating system. However, I propose using ArchLinux ARM for this
project. There are a few reasons for this choice.

1. Arch linux is very minimal which will help us achieving fast boot times
2. Packages are updated constantly (rolling release)
3. Includes a docker package in its official repos. Who likes random
pre-built binaries :)
4. Familiarity with it since I've been using it for long time myself
on servers and laptops

Filesystem
----------

For this setup, I propose using ext4 for the root partition that will
be on the SD card, and btrfs for the 1 TB hard disk that will hold the
data.

The reason for choosing btrfs is its checksum and snapshot support.
Since a lot and important data will be stored on this system, it is
vital to detect when the data went bad and repair them, even if a
single bit flipped [1]. Arch linux comes with systemd monthly timers
that scrub the btrfs partition and repair/report any corrupted data.
Additionally, btrfs snapshots can be used for easy, consistent
backups. First take a read-only snapshot of the data partition, rsync
it to the backup location, delete the snapshot.

As someone else on the mailing list mentioned, Raspberry Pis can
corrupt SD cards if the SD cards are poor quality and the Pi isn't
powered down properly. In order for this to be a reliable system the
power supply must not create any spikes or weird power input during
start up and power down. We don't want bits flipping on the SD card.


Setup without requiring the monitor/keyboard
--------------------------------------------

I think using Bonjour/mDNS technology here is a good solution for
configuring the device. Running avahi on boot will allow other clients
in the same local network to connect to the pi using a friendly domain
name like "owncloud.local". After that they can be presented with the
OwnCloud interface to complete the setup of their instance.


An image with a decent, modern PHP
----------------------------------

We (at resin) have already ported a lot of the official Docker library
images to armv6, armv7 and i386 [2]. Fortunately, Docker has official
images for PHP[3] and OwnCloud[4] so those could be very easily added
in our set of standard images. This means that the Pi will be running
a modern PHP (5.6) with either fpm or apache2. I believe fpm will run
better on the Raspberry Pi but we should probably get some numbers
here.

Those Dockerfiles compile PHP from source so we'll have an as much
optimised build as possible.


Pre-configured caching
----------------------

The Docker images come pre-configured with OPcache and APCu enabled.
Since this will be a single instance installation these look like good
defaults.


Fast database
-------------

Although PostgreSQL and MariaDB are considered faster then SQLite, I
think SQLite is a sensible choice here. My reasoning behind this is
that the there will generally be low load on the service, since it
will be serving ~1 user at home and that RAM on the Raspberry Pi isn't
much (1GB for RPi 2) so going with SQLite will leave some RAM for the
other essential parts of the system (like caches). Lastly, it makes
the software setup quite a bit simpler since again this is the default
configuration.


A built in backup tool
----------------------

I would go for a simple solution here based on btrfs snapshots and
rsync. The target of the backups could be some secondary attached
storage device or even a remote rsync location. Using some of the more
advanced rsync flags, subsequent backups will only store changed files
on disk while at the same time being stand alone full backups [5]. A
cleanup job could always keep N backups in the past or even implement
a Grandfather-Father-Son scheme [6]. I've been using this method for
the past 6 months and works quite well.



Optional resin.io integration
=============================

Before I start, I'd like to acknowledge that using resin.io for this
project implies that you trust of your software and more importantly,
your data, to the platform. However, I believe there are some unique
benefits regarding UX when using it, so I would let an end user
consider the benefits and make an informed decision.


Operating System
----------------

The operating system for resin.io devices is a custom built Yocto for
the specific board. This operating system is minimal with small
footprint and fine tuned to just bring up the hardware and start
running containers. Building a Yocto linux is a complex and time
consuming process, since it compiles every single package from source,
and this is the reason it wasn't the OS of choice above. But we take
of that. The source code for it is open source [7].

System and application updates
------------------------------

Both host operating system updates and application updates are handled
by resin. Our OS images have a double root partition where one of them
is always active and the other one is inactive. When an update is
available it gets installed in the inactive partition, then a
bootloader switch is flipped and the board is rebooted. This ensures
that updates are atomic, there isn't a single moment where the device
is in a semi-updated state.

Application updates are even simpler. Simply `git push resin master` a
repo containing the OwnCloud Dockerfile to your application will
trigger a build in the cloud and the resulting image will be
downloaded on the device and the old one will be updated
automatically.

Device provision (WiFi)
-----------------------

In the case where someone wants to run his setup over WiFi, setting up
the credentials on a static image is cumbersome. For that reason (and
other configuration options) we have created a dynamic image maker.
Think about it as an HTTP endpoint that has some static operating
system images and then injects supplied configuration into them. Using
this method someone can specify his WiFi credentials when downloading
the image. Afterwards, when he powers up his device, the device will
connect automatically to the internet.

Access over the internet
------------------------

When a device is online and connected to resin, it is possible to
reach it over the internet using a url in the form of
`https://<uuid>.resindevice.io`. Any request there will be forwarded
to port 80 on the device. This means that the OwnCloud installation
will be accessible from anywhere on the internet, not just the local
home network.

Note: Currently there is no bandwidth limit there but there might be
one in the future depending on the usage patterns we see.

Device status
-------------

There is always a device dashboard where the current version of the
code, status, and logs of the device are visible. Also, administrative
actions are possible like reboot and shutdown.

[1] http://arstechnica.com/information-technology/2014/01/bitrot-and-atomic-cows-inside-next-gen-filesystems/
[2] https://github.com/resin-io-library/base-images/tree/master/python
[3] https://github.com/docker-library/php/blob/master/5.6/fpm/Dockerfile
[4] https://github.com/docker-library/owncloud/blob/master/8.2/fpm/Dockerfile
[5] https://gist.github.com/petrosagg/b23cae56de981f51ca12
[6] https://en.wikipedia.org/wiki/Backup_rotation_scheme#Grandfather-father-son
[7] https://github.com/resin-io/meta-resin

Best,
Petros Angelatos


More information about the Devel mailing list