Back up with ease
Nick Peers reveals how to back up and synchronise files over your local network and even further afield, enabling you to stay on top of your data.
Nick Peers reveals how to back up and synchronise files over your local network and even further afield.
There are many good reasons for keeping multiple copies of files stored securely on different machines: backing content up is the obvious one, but it also offers the ability to quickly share files with others without having to jump through hoops such as uploading to cloud storage or emailing zipped files. The best tools are those that are unobtrusive. You spend a little time setting things up, of course, but once done they can largely be left to their own devices, working seamlessly in the background continually monitoring folders and uploading any changes without disruption to your workflow.
The best type of tool for this job is a continuous file synchronisation tool, and our favourite by far is
Syncthing (https://syncthing.net). Read on to discover what it does, why you should use it, and how to set it up and tweak its settings to perfection.
Key features
Why choose Syncthing? First, it’s easy to install, and is designed to be configured from a web-based interface for those unhappy at fiddling about with the command line. It utilises your existing network connections, so there’s no messing about with setting up shared folders or permissions, and it can be used to keep folders in sync with other devices both on your local network or remotely over the internet.
All data is encrypted using TLS and transferred using direct peer-to-peer connections (although relay servers are used to make initial connections – more on that later), so files go directly from folder to folder without touching any centralised cloud servers, and thanks to TLS’S perfect forward secrecy, eavesdroppers can’t listen in to steal your data.
Folders can be shared with multiple people or devices, making it easy to set up resilient backups – for example, you could install Syncthing on your server to act as a central hub for backing up to from all your computers, then add another degree of redundancy by trading storage space with a trusted friend remotely, to ensure a copy of your data is kept off-site. Or you could set up a centralised folder, which you share with one or more people, and swap files quickly and seamlessly.
How to install
Step one is to install Syncthing on the devices that you want to set up shared folders. It can be installed as a ‘base’ distribution, which provides a command-line tool, daemon executable and web-based user interface, and is what we’ll focus on in this tutorial. You can also install GUI wrappers for Windows, Mac, and Linux machines (see the box, right), which turn Syncthing into a complete desktop application, but it’s not really needed.
To install the base application in Ubuntu, simply add the stable channel to your APT sources, then install
Syncthing itself. Omit the first line if you already have curl and apt-transport-https installed:
$ sudo apt install curl apt-transport-https
$ echo “deb https://apt.syncthing.net/ syncthing stable” | sudo tee /etc/apt/sources.list.d/syncthing.list
$ curl -s https://syncthing.net/release-key.txt | sudo apt-key add $ printf “Package: *\npin: origin apt.syncthing.net\ npin-priority: 990\n” | sudo tee /etc/apt/preferences.d/ syncthing
$ sudo apt-get update && sudo apt-get install syncthing
Once installed, type the following and press Enter:
$ syncthing
You’ll see a string of messages appear, and then
Syncthing will open a browser tab to the following web address: 127.0.0.1:8384. You’ll be prompted whether to permit anonymous usage reporting. Make your choice and you’re ready to configure Syncthing.
Basic configuration
The first thing you’ll see on the screen is a plea to set a dedicated username and password for the Syncthing
GUI – this makes obvious sense from a security perspective, so click the Settings button and switch to the GUI tab to set up a username and password, making sure you note it down somewhere secure like your Bitwarden password manager.
Before clicking Save, we recommend ticking ‘Use HTTPS for GUI’, plus – to remotely administer Syncthing from any web browser on any device on your network. Change the GUI Listen Address to 0.0.0.0:8384, substituting 8384 for another port number as advised on the Help link for security and privacy reasons. Once done, click Save.
Close your web browser and you’ll see that Syncthing has tied up your Terminal window. You can safely close the window – Syncthing will continue to run away happily in the background. Going forward, you can shutdown or restart Syncthing from the web interface via the Actions menu.
Auto-start Syncthing
As things stand, Syncthing runs on demand only. A better way is to set it up either as a system or user service that’s always running in the background (as a system service), or when you’re logged on (as a user service). Before proceeding, make sure that you’ve shut down Syncthing.
The system service option is best for servers and single-user desktops. To set up Syncthing this way, issue the following two commands in a Terminal window, substituting myuser with your username:
$ sudo systemctl enable syncthing@myuser.service $ sudo systemctl start syncthing@myuser.service
If you’re on a shared computer, configure Syncthing to only run when you’re physically logged into your account via the desktop or SSH (note the absence of the sudo command):
$ systemctl --user enable syncthing.service $ systemctl --user start syncthing.service
In either event, Syncthing should now be running. Verify it by running the systemctl command again, replacing enable or start with status (such as
systemctl --user status syncthing.service ).
Set up a shared folder
Syncthing is essentially useless without a second PC or device to sync to, so now’s a good time to install it on at least one other device you own. If you’re looking to use it to back up your data, a central server or NAS is a good investment to provide you with somewhere to back it up to – you’ll find tools available for both Synology and QNAP (the latter via the qnapclub.eu repository) in addition to Docker containers.
Log back into the Syncthing web UI on your main PC. It’s time to set up a folder to share or sync. By default,
Syncthing opens with a Default Folder entry – click this to expand its properties and you’ll see that Syncthing has created a /Sync folder in your home directory. To remove this (or any other folder you subsequently link), click Edit followed by Remove and then OK. The folder itself and any files inside it won’t be removed, just the link to Syncthing.
To select a new folder on your PC to sync, click Add Folder and follow the step-by-step guide (right), which will take you through the key steps you need to follow to both set up your shared folder and link it to another device. Once the connection has been made, the two folders will be compared, and files transferred from one to the other until they’re brought perfectly into sync.
If you find that you run into connection problems, or the performance seems slower than you were expecting, check your router’s port-forwarding settings if you’re syncing with devices over the wider internet. If UPNP isn’t enabled, you’ll need to manually forward port 22000/TCP to your PC’S IP address, assuming you’ve not changed this port via the Sync Protocol Listen Address setting under Settings>connections’ (if it says default, it’s port 22000). If this seems okay, check that any firewalls have opened port 22000/TCP for incoming and outgoing traffic (ufw users can simply type sudo ufw allow syncthing and sudo ufw allow syncthing-gui ).
Connecting devices
The step-by-step guide reveals how to manually connect devices, but you can speed up the connection process by designating a remote device as an ‘introducer’. This basically makes it quicker to share folders around your network between devices that have been connected to each other. Visit https://docs. syncthing.net and select Introducer Configuration for a more detailed explanation of how this works.
To designate a device in this way, expand it under Remote Devices and click Edit>sharing tab to tick the Introducer box. You can also tick Auto Accept to automatically share folders added to that device with this one, using your default path as your starting point (change this path from the default home folder under Actions>settings>general tab). Note that ticking the box and clicking Save may not elicit an instantaneous response – it can take a little while before invitations to connect start to pop up.
Advanced sharing tips
Once you’ve set up your shared folder, you can sync it to as many other devices as you wish – not just on your own network, either. If you’re planning to share with a trusted friend, you’ll need to communicate your PC’S identifier (under Actions>show ID) with your friend, either by secure message or by sharing the QR code.
A fundamental aspect of Syncthing is trust: anyone who has your device ID can attempt to connect with you, but that gives you access to their computer in turn, so there’s little reason to worry about hackers trying to gain access to your files. If you’re really worried about security, see Security Principles at https://docs. syncthing.net for more information, as well as what options (such as disabling Global Discovery) are available to you under Actions>settings>connections. Also explore the SSH Tunnelling article, which reveals how to connect two remote machines directly without the need for any relays.
If you do receive a connection request from someone you don’t know, just reject it: doing so automatically adds it to an Ignored list, again configurable through Settings. This ensures you won’t keep being hassled to accept the connection and will also block random attempts to connect from anyone who might somehow get hold of your device ID. Just head to Ignored Devices under Actions>settings to review (and unblock). You’ll also see an Ignored Folders tab, which works in a similar way for folders you’ve rejected connections to.
One last tip: Syncthing adopts a smart file conflict system, where up to 10 versions of a conflicting file may exist. Conflicts are flagged up in the main web user interface to be resolved manually – open the folder and manually review both files (filename.