When I started using syncthing some years ago it worked, but it was still a bit brittle. I ran over time into several situations where the sync would just stop or not start, be slow, some error happened somewhere and the GUI did not show them properly, there was no built in reacting to file modification, just a regular sync interval. It was good enough to be used, but certainly not done yet.
Yesterday I installed it again on my main PC (part of a migration from a different linux distro) and I was impressed: The notifications when adding new devices arrived with a very small delay, syncing just worked, file monitoring is now built in, the error message related to that (inotify limit) was helpful.
Syncthing came a long way and is great software, congrats for reaching 1.0!
I switched to Syncthing from Dropbox a few months ago. While there are a handful of warts, the actual syncing seems to Just Work (TM). I am massively impressed with this, as building even a centralized file hosting service is quite the undertaking. I donated and I'd encourage others to do it as well, link on the homepage: https://syncthing.net/
What I'm thinking of are synced ignore lists of some sort. I think I read in the issue tracker that this is something they plan to change eventually.
And related, sync % reflecting ignored files. For example I've ignored a bunch of transient things like node_modules so my backup server sync is permanently at 70% even though it's actually up to date with all the files I want synced.
I've been using Syncthing for a few months, and it has been bulletproof. Truly set and forget. I was extremely apprehensive when I started, but it convinced me. I wholeheartedly recommend it.
I started shortly after Dropbox deprecated BTRFS and told me
to move to a "better" FS like FAT.
After the initial setup, which wasn't that hard after all,
but took more effort than doing nothing and continue on
Dropbox I was amazed. My setup became better as now my phone
is a real client that syncs files, not just a file browser
that can "star" and cache some files, and there's more
granularity on what's shared.
Funnily enough, the Android app not being (or not having the option) to be a browser and cache is one of the things holding me back from wholeheartedly embracing Syncthing. Different needs for different users.
For those who don't know (including me) could you explain what you mean by deprecating BTRFS? I don't understand how the file system is relevant to a file syncing app like Dropbox.
Neither does.. well anyone. Dropbox decided to drop support for BTRFS and pretty much any other Linux file system that isn't ext4, and to my knowledge still hasn't explained their rationale to this day.
Sharing the reason is either a legal risk, or a loss of competitive advantage risk (some optimization that only works when their whole pool is in particular format).
As of last November, Dropbox only supports ext4 on Linux, NFTS on Windows, and HFS+/APFS on Mac. Couldn't tell you the technical reasons for the change, but they're listed here:
I'm using syncthing to sync three computers. Two slaves connected to one 24/7/365 online master. Unless I'm behind a very aggressive firewall, I can connect to master to sync everything.
It creates a big peace-of-mind. Debian's version was built with iNotify for a very long time, so I never got big CPU spikes. I think iNotify is merged into master last year or so.
I don't remember enabling iNotify earlier than it was merged in master. Did I miss a build flag that enabled it earlier than I should have?
Anyways, I'll use this opportunity to say that it has always been great to work with the guys at Syncthing. They responded quickly to every issue that we have ran into. For example, the switch from FontAwesome to ForkAwesome was done for due to a Debian issue:
IIRC, the package description said something similar to "this is the iNotify version of the syncthing binary", or I may be completely wrong. I need to get previous versions and look to the descriptions of each :)
You might mix that up with syncthing-inotify, which was an external small inotify watcher that then pinged syncthing via its API. It was the recommended way to get inotify support till the functionality got merged.
Maybe, however all my Debian installations are very strict in their repository configuration, and I'm failing to see syncthing-inotify package in Debian snapshots server [0].
...or maybe I'm making all these up from all the things I read while researching syncthing, who knows :)
Regardless of all, it's working very well and it's dependable now, and it's the point I think :D
Addenda: Actually, forgetting all little details about it is a reliable sign of an application working well and just disappears to the background IMHO.
How reliable has it been? Do you mind sharing the number of files in the shared directories and their total size? I am asking because when I was searching for a cloudless Dropbox replacement, some tools would lose some files (or truncate them to zero).
Then I found Resilio Sync and I've been using it for about 3 years now without any problems. Now I have a feeling that Bittorrent Inc will abandon the project after the acquisition. Syncthing is looking promising.
SyncThing has been completely reliable for the past two years across my Android phone, Linux laptop, and a Linux NAS.
I use it to backup photos / videos that I capture on my phone, as well as to sync my KeePass database and Org Mode files which I edit on both my phone and laptop, often concurrently. My largest folder is 46 GB across 9,416 files.
The SyncThing Usage Data dashboard at https://data.syncthing.net/ has really interesting statistics: apparently at the 50th percentile, users are syncing more than 5,300 files and over 12.3 GB of data.
Unless I kill the service accidentally, its uptime is equal to the system's uptime.
I sync documents and wallpapers accross my computers, I didn't see any corrupted or truncated file, it has been completely reliable from that point of view.
Also, as I said before, if the firewall of the place I connect is not extremely hostile, I can connect to it from anywhere in the world (it also has a auto-fallback to Tor). If you want to go completely dark, you can run your own tracker too. :)
I didn't do anything. When I checked the local page (localhost:8384) to see how syncthing is doing, one of the clients had a tor address in the list of "last known" addresses :)
I've been using it with SyncTrayzor on Windows for a few years and have found it extremely reliable. Currently syncing 3 computers with ~ 10,500 files, 600 folders, and 280 GiB. It is set up for LAN only, as I don't have a use case for syncing on the go.
Resilio is already a separated spinoff from Bittorrent Inc. You don't have to worry about Bittorrent abandoning it ... just Resilio itself going under! (Which might be quite likely, since it's so small on its own.)
I liked Bittorrent Sync and later Resilio Sync a lot, but they have been very unreliable when it comes to pricing and licensing, changing both several times.
Their license structure is currently worthless to me. I have a license, but that is only for home use. They only offer subscription licenses for business. Subscription licenses are harder to purchase in some organizations, plus everyone that I want to offer files to needs to buy a license as well. This is a large downgrade from e.g. Dropbox, where one can just hand out download links, open file submission requests, etc.
I've been using syncthing for a few years because I was sick of missing files when going from my laptop to desktop and vice versa. I wanted to sync most folders in my /home directory (documents, photos, videos, code...) While ignoring node_modules folders.
It has worked surprisingly fantastically. I thought I would need a server to use as a "master" but syncthing actually supports merging folders automatically. It does not need a static IP address for either computer and it will opportunistically sync my laptop and desktop when they are both on the same network.
I had no idea before I tried using it that syncthing could support such a setup so well. High praise.
I've seen Syncthing before but I don't think it is exactly what I need. If I'm not wrong, it is a secure p2p protocol and a suite of apps implementing them to synchronize folders between devices, in the way you would do with Rsync.
That means if I have several terabytes that I want to sync between devices, I'd need to have enough space in every device, is that right? (and each device would get a 1:1 copy of the folders). I read that when there are conflicts, the apps create a new "filename".sync-conflict file so you can manually resolve conflicts.
What I'd like is a secure way to backup all my files to some cloud provider (S3?) across all my devices, and have an MRU cache in a folder where I can configure the size (for instance, keep at most, say, 512mb of the files I most recently used). For things like photos, it would be better if the sync thing would get thumbnails from the cloud, so I would not need to download the whole thing just to find photos.
GDrive / GPhotos is the thing I use now. The combo has a lot of bells and whistles and I'd be fine having a simpler product where I could have more (perceived?) control on how things display, when syncing happens, etc (in lieu of other things like the nice A.I. photo search, integration with GSuite of apps, etc.). Kinda like using a file explorer on my local desktop.
Do you know https://github.com/s3ql/s3ql? It mounts remote cloud storage like a local filesystem and promises a bunch of optimizations against network latency, including a local cache. Apart from the thumbnail feature that should fit perfectly to what you are searching.
The biggest problem I have with syncthing is that it can't be used with iOS, and there are no plans to ever support iOS. I don't personally use iOS, but my wife does, so I can't get her phone to automatically sync pictures of our family with my phone and computers.
I suppose the people involved don't use the platform, and I realize iOS is more annoying to develop for than Android, but it's a pretty major downside and it's tempting me to use a cloud provider instead. I'd much rather just pay $15-$20 or something for an iOS app.
Maybe now with the 1.0 release it will be stable enough that a third party can step up and make an iOS app?
If you can find other people who share your desire and present the developers with a clear case for making decent money that way, I'm sure either they or a third party would take up the challenge soon enough. Many of us who have no personal interest in the Apple ecosystem won't support it just because someone asks because of the effort and investment required - being meaningfully paid for it changes the equation.
I can certainly see the need for something like this, though I've seen a lot more folks on another forum talking about ownCloud and Nextcloud as alternatives to Dropbox business plans.
I really dislike that the Syncthing website appears to have mastered modern "we won't tell you what it is" marketing speak. Did I miss something and this is now a game with scoring for speaking around topics?
"Syncthing replaces proprietary sync and cloud services with something open, trustworthy and decentralized. Your data is your data alone and you deserve to choose where it is stored, if it is shared with some third party and how it's transmitted over the Internet."
Oooooooh, new shiny with buzzwords! What's it do? Is it a replacement for Chrome's cross-system syncing? Firefox Sync (which I think can already be self-hosted actually)? Password managers? Maybe hosted Exchange? Wait, it's not?
Oh, wait, it's a multi-system file syncing package. Well why didn't you say so while you were telling me I deserved to have control and blah blah blah. Cripes, it even says "a large number of features have already been implemented" then provides a list that implies that running on Windows, OSX, Linux, the BSDs and Solaris is one of those features.
There are several places where what it does is implied, but to actually get to a statement of "For this guide let’s assume you have two machines between which you want to synchronise files." I have to go to the bottom of the site, click Docs, then either click on Introduction and Getting Started (in the documentation structure) or click on the "getting started guide" link within the first paragraph of the Introduction.
grmbl get off my lawn
Edit: I should note, I'm not talking about the 1.0 announcement on their forums - that's for folks already actively involved with it. I'm talking about the very spare https://syncthing.net
Lately, I find myself looking for the github icon before anything else in a website. I find a github readme much more readable than any landing website.
Also: top-left logos should link to the root (syncthing.net) not to a sub-root (forums.syncthing.net). When I land on your forum site and I want to know what syncthing is, I expect to find the answer by clicking the top-left logo in the header.
Jesus Christ do I know what you mean. It's like actual things don't even fucking exist anymore and everyone is just trying to sell buzzwords themselves.
I used syncthing to implement part of a project at work and it does a pretty good job, it deserves a website that actually tells you what the hell it is.
I know, right? It seems like a great project, but just saying "P2P open-source Dropbox alternative" would convey as much information as the rest of the home page combined.
Yeah, as somebody who loves using Syncthing, I would be very hard pressed to know what it is by looking at the front page.
I cancelled my YouTube Premium, and started seeing ads on YouTube for Android; and one of the first few was an ad for some app called Killi or something. It jumps straight to how to find it on the app store and google play, and then explains how to set it up, all the while you're wondering what on earth it is.
I feel like I missed my best opportunity (pre-social media) to combat this.
IIRC, Nexium (antacid of some sort) when it was first introduced had ads everywhere full of people talking about the "Little purple pill" and how much it had changed their lives. There was nothing in the ads that indicated anything about what it did, just "ask your doctor if Nexium is right for you."
Had I been a bit more connected at the time I think a real push to start talking about "The purple pill that cured my X" would have been amusing, with X being something like uncontrollable flatulence, incontinence, explosive diarrhea, etc. Unfortunately, that was back in the days when social media was LiveJournal and Usenet was already well on its way into decline.
I've used syncthing to keep my music, books and docs all in sync between my home server and my laptop. It does a great job. I have also used it for a few years now in place of plex sync. I have plex keep certain unwatched shows optimized for mobile and I just sync that directory to my laptop and phone so I always have my unwatched shows with me. Plex sync was totally unreliable for this so I moved away from it and syncthing has never disappointed.
Syncthing is great. I use it to sync my password database and some other stuff across my devices and have not run into a single issue. I set it up 3 years ago and haven't had to touch it since. I had no idea it was technically in "beta" all this time.
Does it still consume vast amounts of CPU and memory? A few years back I used it on a few hundred GB and it almost constantly used 1-2 CPU cores. On the server it racked up thousands of CPU hours, i.e. it cost real money to run.
Apart from that I don't recall any major problems - which is very positive for anything related to file synchronization.
I've used syncthing for a few years to synchronize 100k+ files and 10k+ folders. I used to regularly get CPU usage spikes from syncthing whenever it scanned folders for changes. The default operating mode of syncthing used to be "do a full scan every N (configurable) seconds for changes," and you needed to run a separate program (syncthing-inotify for me) to use filesystem watches instead. Those full scans would be cheap if you're synchronizing just a few folders, but it's CPU-expensive to regularly check tens of thousands of folders for changes (it might have been stat-ing every file for modtimes, too, I dunno).
Over the past year or so, file watching (via inotify/etc) has been built into syncthing itself. I enabled watching on all my computers and syncthing's CPU usage has gone down considerably as a result, while user happiness has increased; I spend less time banging on syncthing's pipes and more time doing stuff with the files it synchronizes.
I tried it a few years ago on a mix on Linux and Windows systems and quickly abandoned it for Seafile because of the CPU issues. Any idea if it now supports (reliable) real-time file watching on Windows?
I'd love to start using Syncthing, but the absence of storage encryption is absolutely a deal breaker. As I understand, it was a planned feature quite a while ago, but now it isn't even in the backlog anymore. A pity, really.
That problem is easily solved at other places in the FS stack. Use an crypto overlay like EncFS and sync only encrypted data. Alternatively, even stream encrypted updates to a directory via a gpg job and sync the encrypted blobs. Another layer, but the script would be so simple it'd essentially write itself.
While it would be convenient to have configurable encryption on endpoints, syncthing is composable, and thus this issue can be worked around. If you have a platform that doesn't allow you to compose tools easily like Android, that's limited technology.
EncFS is not perfect in that if your adversary has the opportunity to watch your encrypted files updating on the regular they can (allegedly) break the encryption, but they can't do it from a single snapshot.
It's a decent protection against an airport sniffing your phone traffic while you're on the public wifi, it's not going to stop the NSA if they decide you're a person of interest. In other words, if this is a concern for you EncFS is an acceptable baseline protection.
It runs on your own devices, which should either be trusted or have their own encryption (it's built into every major OS). Why would additional encryption be useful?
The point is exactly running on untrusted devices - get a el cheapo server at some VPS provider and have it backup all your data + serve it from a always-on decent datacenter connection.
I see. So it would intentionally not be a live usable copy. I've been using Duplicity going to Wasabi for that purpose - encryption, versioning, etc. One of my Syncthing nodes is an always-on PC at home, so the files sync with other devices whenever/wherever they have a network connection, which is what your encrypted node would do.
IIRC, Syncthing is decentralized P2P and synchronizes standard tree(ish) filesystems (like most similar projects). Perkeep does away with the filesystem and instead uses a blob-based merkle-tree-like system with good properties for very long term archival (and as a side effect, can also use P2P and multiple encrypted cloud backups easily). So Syncthing can copy preexisting file structures and fits in usual workflows, but Perkeep has some data storage benefits. I don't know how their performance compares.
When you say P2P, it's only in a technical sense of not having centralized servers, right? I don't want to interact with nodes I don't own, for my particular use case.
If your nodes have direct network access to each other, no.
But if two nodes that have added each other's keys cannot reach each other directly (e.g. are in separate private networks/behind firewalls/etc) but both can reach an external relay server, it may act as a dumb pipe bouncing encrypted traffic between them.
There is a public pool of such relay servers (https://relays.syncthing.net/), and by default syncthing will reach out and connect/announce its itself there so other nodes looking for it could contact it via a relay.
You can host your own relay server and configure your clients to use it exclusively: https://docs.syncthing.net/users/strelaysrv.html#client-conf... or if you only want syncing when your nodes are on the same local network you should be able to just not configure any relays at all.
Been using syncthing for a few years - I had used BitTorrent Sync until my default search provider became Yahoo! and I could no longer trust anything BitTorrent Inc was doing...
Syncthing is great - a folder that syncs, I have a VPC in the cloud, so for the data I really care about, it's mirrored offsite more or less in real time in case my house burns down.
I like a lot the announcement: it's very easy to stay perpetually in beta to escape commitment. Their reasons for graduating are sensible and I wish more developers of software on version 0.x since a long time would replicate it.
same, the ST-WebUI is just not doing it for me. I paid for Resilio now and I'm quite happy with it. But I also really wish for Syncthing to become a lot more mature and widely adopted to have as an alternative in case resilio discontinues.
I didn't know Syncthing and after installing it on a couple of machines I'm very impressed!
However, I'm surprised too see that by default the whole interface is available on 127.0.0.1:8384. It is possible to set a password but it's a bit hidden in the settings and the documentation does not insist on it.
This makes me wonder, how bad is it for a malicious program running locally to have access to this API?
Pity that I don't know of any browser that readily supports HTTP over a unix socket (nginx can proxy_pass it, but then you're back to a TCP listener).
But a multi-user install could reasonably lock things down in that way, though the defaults definitely favor "works out of the box on a single-user machine".
On a multiuser system it could be an issue. Other users with access to make HTTP requests could add a malicious endpoint. e.g. on a shared server of some sort. I could see this in academia or some business settings. Threat model would be an attacker with a foothold on one machine gaining access to the documents of another user.
Could be, theoretically, but this would probably be inconveniencing all syncthing users for a negligible fraction of the userbase.
On a shared computer where you perceive your peers as a threat, adding a password to syncthing web UI is probably a minor entry in a long list of things you have to do to secure your processes and data, and to ensure the kernel and programs you're running are trustable (in practice, you can't verify that with 100% confidence; they can be backdoored, out-of-date with known security flaws, etc etc).
Ensuring security in such a malicious (where your co-workers/peers are apparently a nasty bunch) shared environment is so hard that to begin with that I personally wouldn't put any sensitive data there at all.
It defeats user-privilege separate on localhost - I (or an unprivileged daemon) can't get at parts of the filesystem belonging to their user instead of mine, but I can get at TCP ports on localhost that belong to other user's syncthing daemons, and reconfigure their syncthing to send all their data to me.
You assume a machine with a single all powerful user. These days computers don't really work this way, for our own good. A huge amount of effort is put into keeping programs as isolated as possible (unix permissions, chroots, cgroups...). Most systemd units include isolation features enabled by default.
Having an API that can reconfigure syncing being available without authentication for anything running on the machine feels backwards to me. More so for a project that focuses on security.
If anyone from the project reads this, may I recommend either encouraging users to set a password or mentioning the implications on the page about Security Principles[0]?
The real reason behind isolated systemd services is to confine the damage if a world-accessible daemon gets breached.
If your threat model is you can't trust the programs running on your computer, then maybe you should switch to a different OS, for example Debian, and stick to the package manager.
It's not great but not particularly horrible either. It takes the interesting combination of "this machine is locked down to the point it does not allow outbound connections by default" and "this machine has a default syncthing install with no password set and an actively running piece of malware that can't create an outbound connection otherwise".
Pretty rare someone manages every outbound connection but doesn't configure a password on the tool handling syncing all of their data across the network. Still, would be better if it prompted for a password the first time a user visited the page to set up the client.
As others here have noticed, Syncthing has improved by leaps and bounds over the last couple of years.
I use it for syncing my phone (everything!) to a pc, and keepin several ps's up to date with each other. These days, it just works. Works so well, in fact, that I have begun experimenting with up-sync to a production server. So far, things are looking real good.
Is syncthing already able to do client side encryption (so you can host your files on a server without the server knowing the files you are storing there)? I remember some years back it only used encryption during the file transfer.
I assume if bit-rot happens then a conflict between versions happen (as if the file was changed manually), so you know a file got changed and can recover the previous one ? I doubt there is any self healing capabilities
By the way if you're on linux btrfs has self healing capabilities (and any of the cheap 2/4 bay 64bit synology nas can use the btrfs self healing and coupled with raid protect your data from rot). Not a very big expense for a lot of safety.
How could inotity notice that though? Bit rot happens at a different level. Detecting it would require regular full data scans, right? And then I guess all files must be write-protected to actually trigger a conflict instead of syncing the corruption. Hmmmmm.
If going Synology, make sure to go for their 64bit models (I don't think they still sell 32 bits, but unsure); it's needed to support btrfs. Other good brand is QNAP. good quality 2 bay can be found below 300€, 4 bays below 500€.
As the other commenter said you can also built it yourself but frankly configuring Samba and its permissions and updating your mdadm config when changing a drive and all the other things gets boring and borderline dangerous really quick, just know that if you buy a pre-made NAS what you're paying is not really the hardware (you can get better for cheaper if homemade), it's the software.
I like DSM (synology's software) because it doesn't get in your way, it lets anyone use easily yet still offers the advanced options. If you've never had a NAS or don't want to spend hours in config files every now and then, I recommend it.
(as usual the question for devices like this is not so much "how easy is it to set up" but "how easy it to recover if say a drive died and you need to repair your array")
A NAS of course could be a rather cheap system running Linux/FreeBSD with three drives in a RAID-Z1 configuration, with SAMBA server. Then you're not dependent on what fs Windows supports (though I recall there being some work towards supporting zfs on Windows, so perhaps eventually).
Is this a good thing to use to, say, keep two git repo checkouts in sync between a laptop and dev server, to avoid the need to do "deploys" for hacking/testing? Or is that overkill?
Last time I used syncthing was couple weeks ago v. 0.14.54 and it's still hit or miss for me in terms of upload speed. Whenever I'm sharing something with friends the upload speed is completly random from 1Mbit to 20Mbit(my max upload speed).
I didn't figure out how to debug this issue but I've tried even running my own relay just in case but that didn't solve it either.
For those, like me, wondering what Syncthing is, Wikipedia describes it as "a free, open-source peer-to-peer file synchronization application available for Windows, Mac, Linux, Android, Solaris, Darwin, and BSD. It can sync files between devices on a local network, or between remote devices over the Internet. Data security and data safety are built into the design of the software."
I switched from Dropbox->Seafile after Dropbox stopped supporting BTRFS. Seafile is minimalistic like syncthing, but also has an iOS app that will do photo syncing which is important for me.
Unison does one-to-one synchronization (local or remote). It puts more emphasis on reconciling sync conflicts. It can handle symlinks. It isn't really maintained, is it. It has problems on windows. It's written in Ocaml which had its heydays back then for a short period of time. Unison rather is a conventional tool which you call and then wait until it's done.
Syncthing does many-to-many remote servers synchronization (but you can also do 1:1 or 1:M). It leaves reconciliation of conflicts to the users. If you're lucky, a copy of the file in question is created that has as "sync.conflict" suffix. It ignores symlinks (or copies them as is). It is actively maintained & has a much wider user community. It works on windows ... as well as android etc. It's written in Go. Synchting can also be run in the background to watch folders and keep everything in sync as files change. (This actually is the main use case.)
You can use syncthing for almost everything unison does when you want to sync servers, workstations, phones etc. You cannot use syncthing to keep a memory stick in sync. IMHO unison is still better at syncing two directories with frequent merge conflicts. It's a pity it got trapped in the ocaml hole. I personally replaced unison with syncthing for most use cases.
Yesterday I installed it again on my main PC (part of a migration from a different linux distro) and I was impressed: The notifications when adding new devices arrived with a very small delay, syncing just worked, file monitoring is now built in, the error message related to that (inotify limit) was helpful.
Syncthing came a long way and is great software, congrats for reaching 1.0!