I came across syncosync, immediately bought two Raspberry Pi 4 and HDDs and set everything up for use by myself and my brother. So far, everything went smoothly. Great project!
But I am wondering: What performance should I expect when backing up? The Raspi is connected to my PC via Gbit-LAN. With an sftp client I get ~40 MB/s write speed. The HDD is a 2.5" USB3.0 harddisk which should not be the limiting factor here. With restic I then get only ~20-25 MB/s backup speeds, which I find a bit disappointing.
By going to 64 bit Raspberry Pi OS alone they saw significant speedups. Some mild overclocking (optionally, I guess) could help further.
Or do I have some malfunction here and things should actually be faster? (It’s not the virus scan under Windows, I tried deactivating it and it made no difference.)
Little update: Using the “sosamdin” shell login I can see that there are intermittent “D” markers visible in the state column in htop. So at least some limitation by the HDD seems to be there.
Tim, perhaps you are the first syncosync user outside the syncosync team at all So we are glad you are here…
Thank you very much for the link to speed up file transfer. From what I read there I’d say it is really worth giving it a try. Now in principle you could set up your own syncosync from scratch for raspian 64, what is needed is shown in custompios setup for syncosync image:
There is only one armhf binary - which we do not even use right now - hd-idle, which is provided by the syncosync repo also. Dunno if armhf is supported? I will try it the next days. If it works smooth, I would even suggest to switch completely to 64bit. There won’t be so many 1GB rpi4 out there and even those may still work, I will test.
TBH for me the backup speed was not so important, as everything is working in background and automagically. BUT: it is really important for disaster recovery, as every minute could count in this case.
While testing two rpi4 with the remote recovery mode in my LAN, I reached 300MBit … so, I am really curious how much faster it will get!
Regarding SFTP speed, I do think it would be nice if it were indeed faster on 64 bit OS. I agree that after the first full backup the speed becomes less relevant - except for recovery. But if switching to 64 bit OS gave a speedup “for free”, that would still be great. Unfortunately, I do not have the time to try and build it myself, but as far as I understood you are going to test it.
I was also wondering whether backup speed is limited by competing access to the HDD for backup and sync to partner. Then an option to only start syncing to the partner when sftp transfers of local users have finished could be nice.
As you mention hd-idle: It would indeed be great if you could include an option to spin down the HDDs after a certain idle time. For my personal use-case I think the HDD could idle most of the day, which would save power, mean less noise and probably also longer life for the HDD.
So, keep up the good work, I really like this project!
Cheers,
Tim
PS: I have some comments/feedback on the backup clients you recommend for Windows. Shall I share them (in another thread) or do you consider this offtopic?
Haha, that’s good, so guerilla marketing does work
I will try to go for the RaspiOS with 64bit the next days. Stay tuned. As you can easily switch the SD card and restore the configuration from the HDD, no problem in continuing to use syncosync.
While nothing is impossible, this would be not really an easy one: the sync to the partner is done via lsyncd while sftp/rsync is started via sshd. Both are only started by the syncosync core. So, yes, the core could check regulary if there is some sftp /rsync access and stop/start lsyncd with some hysteresis around. On the other hand, bandwidth restrictions of the WAN wont use too many ressources for the sync to the partner… so we should rethink if this is really necessary and put it somewhere on the parking lot…
Yes, this is something more important. The first version of syncosync (now almost 6 years ago) was using banana pis and had therefore no USB to SATA converters in between, which are sometimes PITA. So from the experience there, there were already 3 different ways to stop a HDD:
well known hdparm
hd-idle (which is different)
scsi_stop (from sg-utils)
Only the last did stop my externe 2.5 5TB Seagate Desk Expansion HDD. Unfortunately I have not found a way to find out if the Seagate Disk is running or not…
But yes, I put it on the todo list and will have a look at it.
Thanks please spread the word
Not at all! I have just started a new category backup clients. While right now recommending Duplicati is always save (and good IMHO), it would be great to have other clients also tested well.
ok, 64bit syncosync seems to look fine. And from what I see, it is around 30% faster, looking at the network graph while syncing between two boxes at the same switch.
I will need some more changes (just noticed that still buster repo is installed on bullseye machines etc.) and then I’d say we should completely switch over to 64bit for raspi. The good thing: it prevents people from using rpi3 and blaming syncosync for terrible slow usb2 performance
Mem occupations seems totally acceptable:
root@syncosync-0002:~# free
total used free shared buff/cache available
Mem: 1894300 193116 40556 1192 1660628 1630368
Swap: 102396 256 102140
At the same time, I checked if overclocking to 2000MHz would change anything. Interesting enough: no. (see comparison above)
To update: just flash the image on a SD card and start the system with this. You will be requested if you want to restore the configuration from the attached HDD, which you should of course do.
Perhaps is save to remember the IP address, as the system will have for the time before restoring the configuration again have the hostname “syncosync”.
Ok, so I flashed the new 64 bit image and rebooted from that. I can see that the machine is up and has gotten the same IP address in the list of network devices of my router. But I cannot acccess the web interface.
What do you mean by “you will be requested”? I was assuming that this was coming up if I open the web interface. But as I said, that is not accessible.
Ok, I think there is a problem with the 64 bit image. I flashed twice just to make sure. Result was both times that the machine booted and got an IP address via DHCP (I could ping it). Also, I saw that the partition had been enlarged to claim the full SD card. But that was it. No connection possible via ssh or web interface.
I then reflashed the original 32 bit version. It booted and I could access the web interface and in there initiate the restore of the configuration from the harddisk. It says this will take a while, but I assume it will go just fine.
So I guess you need to have another look at that 64 bit image. I can try again if you have a new one.
Ah, one more detail I noticed: The hostname my router displayed was not “syncosync” but “raspberrypi-2”. Not sure if that is useful information.
you are right, I should have had a better look at the build pipeline. My homemade 64bit image did work… I will upload this right now and check for CICD the next days… sorry for the inconvenience
I have flashed the new 64 bit image and booted my box. It came up fine and I could access the web interface. But when I started the import dialog and put in my admin password (I am sure it was put in correctly, tried several times), I got this:
So I thought “hmm, some problem that the 64 bit version cannot import the 32 bit configuration” and re-flashed the 32 bit version. But now I get the same error also on this one!?
Configuration import had worked fine yesterday after my failed attempt to go to the initial 64 bit version and flashing back the 32 bit version. But it does not work anymore today!? That is confusing and unsettling. I had noticed that after configuration import yesterday I could not login via shell as sosadmin and had thus once more set the admin password again through the web interface (but to the same password I had used before) in the hopes of fixing that issue. I still could not log in immediately afterwards. Only after I had once logged in as “tim” I could then also login as “sosadmin”.
I suspect that in one of these steps something broke successive configuration imports!? Could that be? That would be important to fix as there is otherwise a risk to lose data when updating the syncosync image.
Good, I successfully rebooted both boxes now with the 64 bit image. (I had initially thought it didn’t work, but on one of the machines I had made a mistake and the other one got a new IP - so false alarm.)
I guess I will install everything once more from scratch and also do a full backup again. I will see then also if the problem with the overflows goes away. In that case maybe even the regular restart of lsyncd is no longer necessary as a failsafe?
By the way, another quirk with the configuration import: The SSH host key changes. That is not nice because backup clients will then throw error messages and one has to update the known_hosts file on each of them.
Ok, I have set up again everything from scratch and restarted a full backup. What I already see now is that the backup speed is much better than it was with the 32 bit version. I will keep an eye out for the OVERFLOW messagen and report back.
The only major thing that I now still have on my mind is the fact that configuration import failed for me when trying to do so a second time. I think it would be important to track that down and fix it. Everything else I noted is just cosmetics or of minor importance.
Regarding the workflow for partitioning: yes, I think we need to inactivate some buttons
But I am really puzzled about the configuration password. This is even automagically tested and works. but TBH only with the default password I guess, you do not have the logs for this?
config backup download will be added soon, so you do not have to wait for a mail.
Sorry, no I did not keep them. The funny thing is that the first restore worked, but the second one did not. Only thing I can think of is that I had set the (same) admin password again via the web interface.
Sounds good. I realize I had gotten an email with a configuration file at some point, possibly I could have used that one to restore, but I did not try as I anyway was inclined to start fresh from scratch. But the configuration file was from the initial setup, so I am not sure if it would still have worked for the second restore, as something clearly must have changed in between.
Hmm. From my understanding how the password encryption works, this shouldn’t change it. The password salt is contained in the strange configbackup filename. In a next release, I think we will change that to a better readable zip file with a manifest inside.
I think I have to look at the houskeeper again: in the beginning of the project, it did send a new configbackup after a change of the config, now it does send it with the weekly update (or what was set). I think best would be to combine both and to make the latest configbackup downloadable.
The basic idea is, that after every config change (also adding/modding accounts is such one) a configbackup is automatically generated.
We are continuing to work on everything, but as all of us need to make some money for our living, this may take some more time.
So we are glad you are here…
Please let me know once it is available, I will want to switch immediately.
I will keep an eye out for the OVERFLOW messagen and report back.
Only thing I can think of is that I had set the (same) admin password again via the web interface.
