These are well know protocols for communication and are used
- Emule
- Gnutella
- Bittorrent
- Usenet (NNTP – Network News Transfer Protocol)
- Direct Connect
It is interesting to known the background and how they are implemented…lets take a deeper dive.
Emule
Emule and eDonkey2000 use the eD2k protocol or network – I am not sure if it is a protocol.
eD2k stands for eDonkey2000 network.
eDonkey2000 client was the original client for Metamachine. MetaMachine was the original closed source server.
eD2k was created in 2000, the open source emule client in 2002 (May 13th, 2002).
The eD2k network is decentralized – no central hub for the network. Files are not stored on a central server but are exchanged directly between users based on the peer-to-peer principle.
The eD2k protocol is not formally documented
eDonkey client programs connect to the network to share files.
eDonkey servers act as communication hubs for the clients, allowing users to locate files within the network.
Hash identification – files on the eDonkey network are uniquely identified using MD4 root hash of an MD4 hash list of the file. This treats files with identical content but different names as the same, and files with different contents but same name as different.
In 2004 – eD2k overtook fasttrack as the most widely used file sharing network. In 2007 bittorrent overtook eD2k.
The main server software used for eD2k network is known as Lugdunum server. It was created by reverse engineering edonkey protocol and redesigned from scratch.
Lugdunum server software is gratis, but not open source. The stated reason for not opening the source is to prevent the easy creation of fake servers and to prevent attacking the server itself.
The creators of eDonkey went on to do some interesting things in tech: Jed McCaleb and Sam Yagan.
DHT – Distributed Hash Tables – significantly improved searching. Instead of having to search every node in the network O(n) a shared hash table can decrease that to O(logn) or something so only 21 nodes need to be queried to find if the file exists on the network.
Gnutella
14 March 2000, the client named Gnutella became available.
The Gnutella network is a fully distributed alternative to such semi-centralized systems as FastTrack (KaZaA) and the original Napster.
Limewire is probably the most famous Gnutella client.
It has nothing to do with GNU – Gnu’s is not Unix. They have their own peer-to-peer network Gnunet.
On October 26, 2010, the popular Gnutella client LimeWire was ordered shut down.
Bittorrent
BitTorrent is a communication protocol for peer-to-peer file sharing (P2P).
Bram Cohen released the first client on 2 July 2001.
Cohen wrote the first BitTorrent client implementation in Python.
Up until 2005, the only way to share files was by creating a small text file called a "torrent", that they would upload to a torrent index site – it had no search or peer exchange.
In 2005, first Vuze and then the BitTorrent client introduced distributed tracking using distributed hash tables which allowed clients to exchange data on swarms directly without the need for a torrent file – a magnet link.
The BitTorrent protocol can be used to reduce the server and network impact of distributing large files. Rather than downloading a file from a single source server, the BitTorrent protocol allows users to join a "swarm" of hosts to upload and download from each other simultaneously.
Newsgroups (Usenet)
Also known as news servers. This is a client-server protocol – not peer-to-peer.
Uses the Network News Transfer Protocol. The specification was released in March 1986.
Port 119 was reserved for NNTP. News over TLS is usually on port 563.
The specification says to be used with the command line interface but many use gui’s (graphical user interfaces).
While newsgroups were not created with the intention of distributing files such as pictures, sound and video, they have proven to be quite effective for this
More useful is that users download from a local news server, rather than from a more distant machine with perhaps limited connectivity, as may be the case with peer-to-peer technology
As workarounds, codecs such as Uuencode and later Base64 and yEnc were developed which encoded the binary data from the files to be transmitted (e.g. sound or video files) to text characters which would survive transmission over Usenet.
That is why many files have the yEnc extension.
Usenet is distributed among a large, constantly changing set of news servers that store and forward messages to one another via "news feeds".
There are peers at a server level.
I tested it out with a local newserver (axxess): news.internet.co.za and got a lot of stuff. Note that you have to be an axxess subscriber to use their news server. Much of it is unfortunately spam from 2008. I found the comp.lang.python newsgroup and it seems to still be used. You can then read the articles.
Direct Connect
A peer-to-peer text based protocol. There is a central hub of information (centralized).
There is no official specification of the protocol, meaning that every client and hub (besides the original NeoModus client and hub) has been forced to reverse engineer the information
Transporting downloads and connecting to the hub requires TCP, while active searches use UDP.