[owncloud-devel] wanted behaviour on chunking big files and upload them

Daimonion tschmi at googlemail.com
Thu May 12 16:22:18 GMT 2016


Hello First Post here in this mailing list.

My brother in law setup a debian server with owncloud 8.2.4 and we ar eboth
using it as a backup system. 

I'm using the OwnCloud Client 2.1.1 Build 5837 and i'm syncing files via
internet to this ownCloud instance.

At the moment i'm upload a bunch of really big Files (4,3/7,8 GB (600GB in
total) ) as i a want to store my system backup automatically outside my
house.

So long everything is good with the setup. The Owncloud runs very stable and
i synced many files (little ones (kb) and also big ones (1-2GB)) before the
really big files (4,3/7,8GB) without problem. 

But with this files i ran into a problem. "24h forced disconnect of dsl
line". Every night my dsl line gets disconnect (Deutsche Telekom) and
disrupts the upload. No Problem for Owncloud. It logs a failure "write not
possible" and as soon as the connection is online again it resumes upload.
Let you explain an example.

The Big Files are splitted in 1600 chunks. So let's say i start an upload
from a new file in the evening. Then maybe the first 1000 chunks (3 at the
same time) will be uploaded until the forced disconnect disrupts the
internet connection. The client fails on chunk 1000,1001,1002
After reestablishing the connection the client resumes the upload with chunk
1000,1001,1002. Fine. But if it reaches the last chunk 1599 it will start
over with chunk 0 and it gets more worse. There are 3 parallel uploads, and
the first time the file was uploaded all 3 upload sockets were used by the
same file. Now as it started over from chunk 0 it uses only 1 upload socket.
the 2 remaining sockets will be used by other files (even the same big size
(7,8GB)).

Now as there remains just 1 upload socket it will take ages to upload the
remain chunks. And there is no detail how much chunks have to be
uploaded......


What is the intended behaviour on forced interrupt while chunked uploading
and is there anything we can configure that a forced disconnect doesn't not
ends in eternally upload the same chunks again and again?

Is there a possible way (server or client side) to look which chunks are
uploaded correct and which chunks have to be uploaded to complete the file
on the server?


Thanks in advance

Regards
Daimonion



--
View this message in context: http://owncloud.10557.n7.nabble.com/wanted-behaviour-on-chunking-big-files-and-upload-them-tp17249.html
Sent from the Developers mailing list archive at Nabble.com.


More information about the Devel mailing list