Rendered at 05:06:28 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
internet2000 59 minutes ago [-]
Starting with robocopy is nuts. I'd have started with rsync and just let it run overnight. The peace of mind that it'd do the right thing and not needing to research flags are more than worth any speed tradeoffs.
kalleboo 2 hours ago [-]
The Synology web UI lets you mount a remote SMB share and copy files directly in their file manager without making the round-trip via your PC, I imagine this would have been the fastest method without getting bogged down in command-line tools.
teekert 23 minutes ago [-]
I didn't read the whole thing, but couldn't you just rsync over ssh from either end?
Your suggestions is also very nice, you can just turn off your own PC and wait for it to finish.
ZPrimed 18 minutes ago [-]
Synology actually supports the old (non-encrypted) rsync protocol, too. On a trusted LAN, it runs much quicker that way than with rsync-over-ssh...
I just finished slurping all of my files off an old Synology and onto a TrueNAS server. Unlike the article here, my Syno is 2x1Gb in a LAG, so I was really only getting ~1Gbps at max.
One disappointment with TrueNAS is that it doesn't support SMB-MC unless you put each interface on a separate subnet (which is a strange limitation because Samba itself doesn't seem to suffer from this, since synology handles it fine)
Neywiny 2 hours ago [-]
Sometimes when we Linux for a long time, we run into decades old friends: grep, sh, bc, whatever. Not uncommon to run a command that first released 50 years ago. Rarely on Windows do we get that. As much hate as windows gets, at least they left in more than just explorer drag and drop.
If the author is reading, it would be nice to clarify that the 10g network comment is irrelevant. I was under the impression at least 1 of the devices would have a 10g link, but at least 2 of the 3 didn't. It was a bit confusing, but it's also very late where I am.
joshka 2 hours ago [-]
My interpretation is that the setup is 10GbE UNAS Pro 8, 4x 1GbE Synology DS 1520, 1x 2.5GbE Windows host.
ZPrimed 20 minutes ago [-]
If that's the setup, the only way anything would run more than 1Gb is if the Synology is doing SMB-multichannel (which if they setup 4x1Gb LAG, then it's not SMB-MC).
[edit] reading the article, they do have the Syno setup for SMB-MC and that's the only reason this wasn't stuck at ~120MB/s
bananamogul 2 hours ago [-]
"Ideally you'll find more value in the comments as Hacker News folks and Windows experts will drop in with better tools and strategies."
I don't know if this is better, but...an alternative is rclone, which uses rsync under the hood. It can also do parallel jobs and is restartable on any failed partial copy.
When I've had to do large Synology-to-Synology moves, that's what I've used. The multiple rsyncs in parallel really helps if you've got a lot of small files.
randerson 2 hours ago [-]
Moving Synology-to-Synology, I was surprised at how easily I could just move the original HDDs (in their original order) from old enclosure to new enclosure, and it just worked.
UltraSane 2 hours ago [-]
I consider it much better because rsync can detect bit errors and robocopy can't.
tamimio 2 hours ago [-]
The issue with rclone compared to rsync for existing files is it will copy the whole thing again, where rsync uses something called delta transfer and only transfer the change. But if you are doing one time cloud to cloud transfer then yeah rclone is the gold standard.
> rsync under the hood
Is it? I think one is written in C (rsync) and the other is in Go lang (rclone)
jaden 1 hours ago [-]
I had the same reaction. rclone is not built on rsync. It's an independent implementation in Go, inspired by rsync.
c0nsumer 2 hours ago [-]
I love this kind of step-by-step solving post, but it bothers me that the author didn't track down what the JPEG in the Alternate Data Stream (ADS) was for. Something put it there... Might it be a surprise when it ends up not being there later?
Also, it kinda sounds like the UNAS might not be supporting ADS' for some reason? That alone could be a problem later on...
joshka 2 hours ago [-]
Fairly reasonable explanation is album cover art.
mmetzger 1 hours ago [-]
The moment the /Z option showed up I figured there'd be an issue. I have to copy multi-TB files often using Robocopy. We originally used /Z because "Well, if it gets interrupted, I'd obviously want to restart from where I left off..." I accidentally left the switch off one day and noted it completed in less than half the normal time (of using the /Z flag).
In other words, yes, it may be restartable, but if it takes more than twice as long, just recopy the file(s) if needed.
para_parolu 1 hours ago [-]
Has anyone found a solution to lack of encryption in unas?
gregoryl 1 hours ago [-]
It has encryption.
tiew9Vii 2 minutes ago [-]
From the benchmarks I’ve seen there’s a significant throughput degradation with encryption enabled on the UNAS’s due to limited cpu.
mmastrac 2 hours ago [-]
rsync tends to be a bit tougher on smaller CPUs. scp is generally better for those cases if you don't mind hashing afterwards to confirm everything made it without bit flips (in theory it shouldn't happen, but I've definitely had corruption during transfers like these). I personally prefer that method because you check files that are out of page cache and at rest on the filesystem.
If you want to get the advantage of SMB multi stream copies (in rsync*), look for LACP or other options. Not sure if Synology does that though.
I'm honestly surprised that SMB was faster here. I've never had luck getting it to perform well, so this is a bit of a shock.
luma 2 hours ago [-]
LACP does not work well with SMB multipath and will generally prevent you from seeing the benefit. You want IP addressed adapters for each link on both sides of the conversation.
sirjaz 2 hours ago [-]
It shouldn't be shocking SMB is very fast and more secure than NFS. Hell it supports running over quic now
joshka 2 hours ago [-]
It really feels like rsync should have a multi-path setting that would have sped this up across multiple links (but doesn't because noone has made it yet).
jbird99 2 hours ago [-]
I use bbcp for that, it supports multiple streams.
Your suggestions is also very nice, you can just turn off your own PC and wait for it to finish.
I just finished slurping all of my files off an old Synology and onto a TrueNAS server. Unlike the article here, my Syno is 2x1Gb in a LAG, so I was really only getting ~1Gbps at max.
One disappointment with TrueNAS is that it doesn't support SMB-MC unless you put each interface on a separate subnet (which is a strange limitation because Samba itself doesn't seem to suffer from this, since synology handles it fine)
If the author is reading, it would be nice to clarify that the 10g network comment is irrelevant. I was under the impression at least 1 of the devices would have a 10g link, but at least 2 of the 3 didn't. It was a bit confusing, but it's also very late where I am.
I don't know if this is better, but...an alternative is rclone, which uses rsync under the hood. It can also do parallel jobs and is restartable on any failed partial copy.
When I've had to do large Synology-to-Synology moves, that's what I've used. The multiple rsyncs in parallel really helps if you've got a lot of small files.
> rsync under the hood
Is it? I think one is written in C (rsync) and the other is in Go lang (rclone)
Also, it kinda sounds like the UNAS might not be supporting ADS' for some reason? That alone could be a problem later on...
In other words, yes, it may be restartable, but if it takes more than twice as long, just recopy the file(s) if needed.
If you want to get the advantage of SMB multi stream copies (in rsync*), look for LACP or other options. Not sure if Synology does that though.
I'm honestly surprised that SMB was faster here. I've never had luck getting it to perform well, so this is a bit of a shock.
https://docs.nersc.gov/services/bbcp/
for example, Immich is a huge upgrade to Synology photos. I didn't know how much i was missing out.