Horrible SMB Small File Performance
I have a DS1815+ with 7 4TB WD Red drives in SHR2 with btrfs. The problem is that copying small files (my test set are 128KB - 512KB) is extremely slow. Synology support has so far been unable to provide any solutions (I've been working with them for a week now trying to fix this).
My test data is around 6,000 files, 3,000 of which are 128KB in size and the other 3,000 are 512KB in size. This is a typically set of data that I work with every now and then (they are png and pgm images if it matters).
I can put this data on a USB drive, connect it to the Synology and copy it using DSM File Station to my test share in about 3 min. During this transfer, the small files copy at about 8MB/s and the large ones at 30MB/s. This isn't great in today's SSD world, but I can live with it. I know small files are a problem.
I can put this same data on a client machine and copy to one of my Windows server shares in about the same amount of time through SMB3 (confirmed that SMB3 is being used by using Get-SmbConnection powershell command).
But if I put it on my client machine and copy to the Synology share (the same one used during the test in DSM with direct attached USB drive) then the copy takes about an hour and a half. The small files transfer at <100KB/s. The larger ones are faster, but it still takes an insane amount of time. Less than one file per second. Typically the transfer starts out fast (as reported by the Windows file copy dialog) but then within 5 seconds drops to the above speeds.
Using NAS Performance Tester 1.7 program shows that with 1MB files (the smallest I can specify) it goes fast for a few files (30MB/s) and then slow for a few (<1MB/s). It kind of alternates between the two. It seems like there is some caching done somewhere or something ... not entirely sure what to think of these results.
Using ROBOCOPY on the client machine has no problems. It finishes in about 3 minutes (same as the direct on-device copy).
Connecting a client machine directly to the Synology (ethernet between the two) shows the same horrible performance so it's not the network hardware at fault.
Connecting to Synology using FTP shows good performance (no problems). So this seems to be isolated to SMB.
Connecting an external SSD drive to the Synology with USB3 and sharing it over the network and using that as my test share shows the SAME poor results. So the problem is not my disks or SHR2 volume (or btrfs).
So, obviously my client machines are fine since they can all copy to the Windows server at full speed. And my network is fine since the clients can copy to the Windows server over the same network and the problem exists even when connected directly to the Synology. As for why ROBOCOPY can bypass this problem I have no idea. FTP doesn't have any problems so it must be Synology SMB that is to blame. I
Other details that might be relevant: * Synology is connected to a Windows domain. * SMB is configured to use SMB2 with Large MTU or SMB3 only (changing this to regular SMB2 doesn't help the problem) * I tried disabling transport encryption. Had no effect. * IPv6 on Synology is disabled (this sped up browsing of the shares but not the copy speeD) * the test share I'm using has no encryption or advanced data integrity enabled. * during the copy the server resources are fine - CPU is maybe 25%, Disk and Volume utilization is low (5% with some spikes every now and then to 10%).
Anyone have any ideas? Synology support has been pretty useless so far - they just keep recommending things from the support page about slow performance. Even though I was clear I had already tried all those things before contacting them.