[owncloud-devel] GSoC Proposal for Large File Sync

Jos Poortvliet jospoortvliet at gmail.com
Tue Mar 22 19:02:04 GMT 2016


On dinsdag 22 maart 2016 11:31:39 CET Tomaz Canabrava wrote:
> On Tue, Mar 22, 2016 at 11:16 AM, Klaas Freitag <freitag at owncloud.com>
> 
> wrote:
> > On 21.03.2016 23:58, Tomaz Canabrava wrote:
> > 
> > 
> > 
> > Hi,
> > 
> >> I can work on a proof of concept for large text files and virtual
> >> machine images (wich would already be a win-situation for some users)
> >> and then focus on *some* of the hard to sync files (like powerpoint
> >> presentations) and see what I could get.
> > 
> > I do not think you should consider the file type at all. Just try to
> > implement the zsync based approach I'd say, and just for the chunked
> > upload mode.
> > 
> > Raw steps:
> > 
> > 1. on the client, chop the file in chunks and create a list:
> >   Number of chunk     start-at-byte    end-of-byte   Checksum?
> > 
> > 2. send this list to the server to get the servers checksums
> > 3. While waiting on the server list of checksums, calc the client
> > checksums
> > 4. compare the lists once the both are ready and decide which need upload
> > 5. upload the chunks that changed.
> > 
> > The trick is in the cutting of the chunks. The amount of chunks that do
> > not change can be increased by picking clever boundaries.
> > 
> > This project requires both client and server work.
> > 
> > Please do the server work based on what is described in the blog series
> > about the new chunking API, there is a branch with basic implementation of
> > that here:
> > https://github.com/owncloud/core/pull/20118
> > 
> > Makes sense?
> 
> Makes sense.
> 
> I have a university test today night so I don't know what time I'll get
> back home, but  I'm already installing and compilling owncloud from source
> here to poke around things.
> 
> What blog series about the new chunking api you refer? owncloud blgo or
> rsync's?
Here's Pt 2 of that blog series he mentions:

https://dragotin.wordpress.com/2015/07/10/owncloud-chunking-ng-part-2-announcing-an-upload/

> Tomaz
> 
> > regards,
> > 
> > Klaas
> > 
> > 
> > 
> > 
> > 
> > _______________________________________________
> > Devel mailing list
> > Devel at owncloud.org
> > http://mailman.owncloud.org/mailman/listinfo/devel


-- 
Disclaimer:
Everything I do and say is based on my view of the world today. I am not 
responsible for changes in the world, nor my view on it. Everything I say is 
meant in a positive and friendly way, unless explicitly stated otherwise.
find me on blog.jospoortvliet.com



More information about the Devel mailing list