Open source sharing
P2P Learn how to give and take in the world of peer-to-peer filesharing Alexander Tolstoy helps you get the most from BitTorrent, showing you how to become a leecher, a peer and a host of encrypted precious files.
Alexander Tolstoy helps you become a leecher, a peer and host to a small pile of precious encrypted files.
Transferring large amounts of data between remote computers can still be challenging for many Linux users. Sometimes you just don’t want to write it on an external hard drive or upload to a cloud-based service. Both ways take much time and effort while not being very safe. In this tutorial, we’ll examine methods for sharing files using the peer-to-peer (P2P), which is a top-notch technology for interchanging data.
The big entertainment corporations may not like the technology due to possible copyright laws infringement, and we certainly don’t endorse sharing files that you don’t have a right to, and start to see P2P as something that can be used for good more so than for bad. After all, you may just want to share a Linux distro ISO or a homemade movie for your family rather than the latest blockbuster. We’ll take a look at Direct Connect and BitTorrent software implementations, both of which have state-of-the-art desktop clients for Linux.
Connect to a hubs
There are dozens, if not hundreds, of local hubs in almost all countries. The idea of a hub is to ease data interchange in local communities by connecting to a nearby hub. As long as the data flows inside a local segment of your service provider’s network, the upload and download speeds can be very high. In this example, we’ll learn how to get connected to a hub and share some files from your computer. To do so, we’ll need a DC (direct connect) client, and we’ll use EiskaltDC++. This is a feature-rich and flexible application that was derived from another widely known DC software, Valknut.
When you launch EiskaltDC++ for the first time, you see a clean and almost empty interface with colourful buttons on the toolbar and a categories list on the left side. The hubs ecosystem exists by ‘give and take’, so before we connect to a hub, let’s choose what we want to share. Go to Tools > Preferences and then proceed to the Sharing section. Here you can add a directory that will be exposed to other users once you connect to at least one hub. Next, examine your settings in the Connection sections. Use the ‘active’ mode if you have a direct Ethernet connection with a static IP address and no last-mile routers. If you connect to a home router that broadcasts data to all your in-house devices, you will need to choose one of the firewall modes. Almost all routers support Universal Plug and Play (UPnP), which means that you need to tell your router to redirect all incoming packets to your PC. Consult your router’s manual on how to do this in detail, but the general concept is simple: tell the router (e.g. 192.168.1.1) to forward all packets to your PC (e.g. 192.168.1.35) to a given port (e.g. 3000). The port number can be random, but the value in your router’s settings should match the one in the EiskaltDC++ settings. Finally, hit ‘OK’, go to Hubs > Quick Connect and enter the hub’s address, starting with the dchub:// prefix.
You can easily search for your local hubs list and choose whatever hub you want, provided that it resides inside your service provider’s network or has access to the internet.
In the latter case, your connection speed will be limited by your regular internet download rates. Most hubs have a defined minimum share requirement to let you in.
EiskaltDC++ needs some time to get hashes of your share files, so you may need to wait a little while before everything settles down. The great feature of any DC client, including
EiskaltDC++, is that it’s very social. The main area of the application’s window is dedicated to the IRC-style chat that greets you with welcome information, rules and registration details. Once registered, you can take part in conversations and connect to the hub with your nickname and password. You can add any hub to favourites and specify your credentials there.
Use the ‘Search’ button on the toolbar to go to a file search dialog and enter your query there. The application will look up what other people share and display the list of the results below. Double-click any file or directory to put it in your download queue. Naturally, the machine that is sharing what you’re currently downloading should be online so that your progress will depend on other people’s behaviour during certain times of the day.
Sharing within LAN
If you need to send a file or directory to another computer in your local home or office desktop, you’ve probably thought of Samba, CIFS or NFS. All three methods are suitable, although two of them are historically Windows-specific and require additional skill to implement. We’ll use alternative tools that don’t bring much overhead and offer an easy-to-use way of accessing files from other devices. The first solution is Weborf, a minimal web server designed for rapid sharing of directories. Weborf makes your content accessible via any web browser and shows the directory that you specified as an http:// address. The application is developed together with a corresponding GUI called QWeborf, which saves you from messing with command-line arguments.
In the main application window, go to the Basic tab and select a directory that you want to share and review default values in the Intermediate tab. Here you can change the suggested server port ( 8080 is used by default), enable authentication, turn on the WebDAV mode and adjust other settings. For instance, check the ‘Send directories as .tar.gz’ option if your recipient needs to download everything you share in bulk. If you enable the WebDAV mode, your clients will be able to mount your share via their file managers and access it as if your share was just another local directory. Consider allowing other people to write files on your server by ticking the option below the WebDAV checkbox. QWeborf already includes a miniUPnP server for traversing NAT and bypass your router to share files outside of the local network.
Another very useful tool is Dukto. It doesn’t need any servers, neither does it implement its own web server. Instead, Dukto enables you to share files between any devices within a LAN. The great feature of Dukto is its large number of supported platforms, including all the major OSes for both the desktop and mobile devices. With Dukto, you can share files between Linux, Mac, Windows, Android or even ancient Symbian phones, without any wires or compatibility issues.
The application UI is consistent across all platforms. Just click or tap a destination device in the Buddies section and use the ‘Send some files’ option to select what you will be sharing. You can also enter the recipient’s IP address manually by going with the ‘IP connection’ entry. This way you can share your data outside LAN, provided there are no obstacles on the way (e.g. firewalls). Regardless of the way you want to use Dukto, you need it running on both ends.
QWebord and Dukto are both peer-to-peer network tools that will work even if your internet connection is down—you only need to maintain a LAN connectivity. Another common feature is the ability to access your files from mobile devices. You can set up Samba or configure an NFS storage on your Android device, but, in the case of the above tools, all you need to do is open a link in a web browser.
We’ve already touched on the WebDAV technology. This is a useful extension because it enables you to mount a remote URL as a local (aka FUSE) filesystem with your standard file manager, such as Dolphin or Nautilus. You can enjoy thumbnails, bookmarks and any other features of your file manager, as if you were browsing a regular local directory.
QWeborf allows you to enable WebDAV access to your share and here is how you can access it from the other end.
Make remote files local
In Dolphin navigate to the list of network locations at remote:/ and click the ‘Add network folder’ button. In the new wizard window, which is virtually the Knetattach program, select the ‘Webfolder (webdav)’ variant and click ‘Next’. In the next screen you’ll need to provide connection details. In the case of a public WebDAV share you only need to give server name and port. The required ‘Name’ field can be whatever you want and the ‘Folder’ field can be left empty (meaning that you’re mounting the root of the share). In the case of the Gnome-centric Nautilus file manager, select the ‘Connect to Server’ option and provide your WebDAV share address using the following syntax: davs://user:password@host.name:0000/path
Where 0000 is port number. Again, the unprotected destination will be accessible via the stripped down string like davs://host.name:0000/path
For minimal desktop environments or console mode-only systems, you may want to have a working command line solution for mounting WebDAV directories. Linux has the splendid davfs2 utility that offers a great way of manipulating remote filesystems. After installing davfs2, first, add yourself to the respective user group: $ sudo usermod -aG davfs2 $USER
Next, edit the configuration file to let davfs2 know authentication credentials. Open the ~/.davfs2/secrets file and add a line according to the following template: $PathToMountPoint $USER $PASSWORD
For example: /home/user/owncloud_share user P@ ssw0rd. (Of course, create the destination mount point directory in advance). Now mount your share: $ mount -t davfs http://192.168.1.3/owncloud/files/webdav. php /home/user/owncloud_share
Don’t forget to use the real IP and path. You can do this automatically upon system startup by adding a similar line into you /etc/fstab file: http://192.168.1.3/owncloud/files/webdav.php /home/user/ owncloud_share davfs user,rw,noauto 0 0
Using cloud services is a specific filesharing use case and since we want to access our remote account in the most seamless and natural manner, we can’t skip RClonebrowser ( seeHotPicksp57,fordetails), so here are some practical considerations. As long as this tool browses remote cloud accounts instead of continuous syncing them with your local directories, you can obtain specific files with just a few mouse clicks. The main tree view in RClonebrowser supports uploading and downloading files on demand, but there’s also the ‘Mount’ button. This can be used to make your Amazon S3, Google Drive and other supported cloud-based accounts look as if they were regular directories. The real downloading will only start once you decide to copy a file from that mounted directory somewhere else.
Use BitTorrent
Many Linux users download large files via BitTorrent technology thanks to its simplicity and high speeds. Using a torrent is very social endeavour because your download speed depends on the current number of seeders—people who are sharing data for that torrent. What’s even better, you don’t need to do much to join the crowd of seeders: once you have downloaded all segments of a torrent, you automatically start sharing it as long as you don’t quit your torrent software. Downloading something using torrents is still a one-sided affair, even though you help others to raise downloading speed, so let’s advance a bit further and create our own torrent seed that others can download.
Creating new torrents is a standard feature of almost any popular BitTorrent client. In the following example we’ll learn how to do this with KTorrent, although the procedure is very similar in other applications. Launch KTorrent and press the ‘New’ button on its toolbar (or hit Ctrl+N). The new torrent creation dialog will appear with various options you can set or configure. Press the ‘Open File’ button to select a directory that will be included in your torrent. For the next step, you
need to decide whether you want to use a torrent tracker or not. A tracker is a natural dependency of the original BitTorrent protocol. It’s a third-party web server (usually a forum run by enthusiasts) where you need to register and post your torrent file to get your first leechers (users that are downloading your torrent but haven’t completed it yet).
Traditionally, when leechers open your torrent file for download, their client connects to the torrent tracker to update statistics and get the list of seeders and other leechers and only after that downloading will proceed as a pure peer-to-peer operation. As an alternative to this approach, you may want to go with a fully decentralised torrent seed. In KTorrent tick the ‘Decentralized (DHT only)’ checkbox to enable the DHT (distributed hash table) mode. In this case, peer IP addresses are stored inside a torrent using the BitTorrent infohash as the key. Since all a tracker does, basically, is respond to put/get requests, this functionality corresponds exactly to the interface that a DHT provides: it allows you to look up and store IP addresses in the DHT by infohash.
So, if you choose to create a regular torrent, you’ll need to add at least one tracker URL in the field below. Take note that trackers’ addresses usually differ from regular URLs that you use to visit their sites, e.g. https://openbittorrent.com suggests that you use the http://tracker.openbittorrent.
com/announce address. If your choice is a DHT torrent, go to the ‘DHT nodes’ tab below and enter the IP address of the server that will provide an infohash table for seeders. Naturally, if you want your Linux machine to serve as a DHT node, you must have a static IP address and keep your system up and online all the time. Press OK when you are finished with the settings. You’re now ready to send it to your friends, but keep in mind that they will be able to download the torrent content only if there is at least one seed, so consider staying online with your torrent open in your favourite client application.
Speaking of applications, the selection for Linux is not limited to GUI-only applications; you may want to control torrents from the command line, ( SeeControlTorrentsFrom TheCommandLine,p69,fordetails.)
Decentralise your file sharing
While some Linux filesharing applications have a focus on usability, there’s a different class of tools that provide anonymity and security. Retroshare, an anonymous friend-tofriend network client, is one such tool. It combines the elements of secure Tox- like chat with EiskaltDC++' s file sharing approach and an exterior that resembles an email client. On the first run, you will be asked to create an identity, which constitutes your nickname and a password. Once you are in, you can start exploring Retroshare’s tabs and sections, although they will all be empty. Since RetroShare only connects to your friends, before using it you need to tell it about them. Press the ‘Add’ button on the main toolbar and select the way you want to establish a connection. The default option is to enter the certificate code that your friend has provided you. Retroshare will also generate a certificate on your side, meaning that you will send it back to your friend. As you can see, making friends requires mutual operations on both ends (like real life). Once you have someone to communicate with, you can share some files and receive files from other people. Press the ‘File Sharing’ button on the main toolbar and find yourself in the File Transfers window. Go to My Files to see what’s shared on your side. By default, Retroshare uses the directory inside your profile, which looks something like ~/.retroshare/<hash value>/Downloads. This means that everything that you download automatically becomes available to others, and, if you have many active friends, the number of files in reach will grow exponentially. Still, it would make sense to add some extra content, so go ahead and press the ‘Add Share’ button. In the Share Manager window, you can add additional directories and assign ‘virtual folder’ names to them. You can also manage access flags. Set these if your files can be browsed by friends from specific groups only or allow anonymous access to your content. When you land back at the File Transfers section, you can use the Friends Files tab to browse materials that are available to you.
Due to the decentralised nature of Retroshare, there are various limitations that you should get used to: You don’t authenticate yourself on any server, so to use your account on a different computer or operating system you need to transfer your profile manually. When logging into Retroshare, note the ‘Manage profiles and nodes’ link. It will lead you to the profile manager that enables you to export and import your identities using the encrypted ASC file format.