[owncloud-devel] Syncing with the new Shared model

Olivier Goffart olivier at owncloud.com
Thu Jun 19 10:46:58 GMT 2014


Hi,

ownCloud 7 is about to introduce a new model for shared folder.
Please someone correct me if i am wrong:

- In ownCloud 6, all shared folder were sub-folders of the "Shared/" directory
- In ownCloud 7, shared folders first appears as 'normal' folder in the root 
directory, but the user can then move them anywhere.

This change need to be taken in account by the sync client. One thing is that 
we will show overlay icon over shared folder in the file explorer. But this 
will also have consequences on the syncing algorithm.

Especially the question on what happens when users make change locally in a 
shared directory, or even move that shared directory.

What you have to know is how the client currently detects the move.  The 
client has a DB with the old state. When the client sees a file with a name 
that was not previously in the DB and if its inode is in the DB and correspond 
to a file or folder which is no longer in the local file system, it will be 
evaluated for move. Directories are always moved. Files are only moved if 
their modification time and size are still the same.

The problem is that inodes are unreliable. inodes are quickly re-used. If you 
do  rmdir foo; mkdir bar;  it is quite possible that bar will get the same 
inode as foo had.
On Windows we saw that inodes (FileIndex, really) can randomly change. Perhaps 
when defragmenting or mounting/unmounting drives.

For that reason, moves detection are quite unreliable. They usually work but 
we should not rely on it.

Also, we have special code in the current sync client regarding shared folder.

In mirall 1.6, since the server did not provide any information about what 
folders were read only or not, we were just trying to propagate the changes, 
and then noticing error from the server. Then we would try to recover in case 
of error.

This is how we currently recover:

 - If the user deletes a file in a read-only directory (on his local file 
system), the sync client recover that file. 
 - If the user edit a file in a read-only directory, the sync client download 
the original file back, and save the modified file as a conflict file.
 - If the user add a new file in a read-only directory, the file is kept, but 
the client shows an error saying the file is not synced
 - If the user moves a file from, to or within a read-only directory, we will 
consider that as a delete + upload on the next sync.
 - If the user moves or rename the "Shared" directory itself, we move it back 
to its original location (this was requested by support)

The main idea here is that after the sync (or a couple of sync) the read-only 
shared directory is still in the same state as on the server, no matter what 
the user does.

The behaviour is different in shared directory vs normal directory.  If there 
is an error uploading a file or deleting a file in a normal directory, we will 
retry on the next sync. (vs. restoring the original in a shared directory)


OwnCloud 7 properly report what directories are shared and read-only, and the 
future version of the client can use this information to do a better sync (not 
to try-and-recover as before, but directly recover)

But should we allow to move a shared directory?

Imagine this scenario:
- The sync client is not running (or offline)
- The user delete a shared directory on his file system
- The user create a new directory and fill it with random files and subdirs.
But that new directory just happen to have the same inode as the shared one 
had
- Then the user starts the sync client, and the sync will result in the 
following:
 * the shared directory will be moved
 * all the content of the shared directory will be removed
 * the new files will be uploaded in the shared directory

If the directory was not shared, that would not be a problem because the end 
result is the same. But now the result is that files that were not supposed to 
be shared are shared.
And if the shared folder was read only, it would result that we would also try 
to recover, and the user will end up with a mix of the two directories.

So the question is:  Should we really try to support moving the shared 
directory from the client?
Should we try to get more elaborate algorithm to detect moves in order to get 
this use case properly (tree comparison)?
Should we ignore this problem as a corner case? (IMHO not, because its 
implication are bad enough).

-- 
Olivier




More information about the Devel mailing list