Hardy wishlist, post #4
Okay, you know the drill by now.
Affects: APT
Issue: While apt is fantastic in it's current design, it is still targeted for the one-way client-server topology of the "old internet", if you will. As anyone who tried to download or upgrade on Gutsy release day knows, both the main server and most mirrors, both archive and CD mirrors, were extremely bogged down, almost to the point of completely unusable. While this is great news in terms of Ubuntu's popularity, it's not so great for the users trying to access those servers, nor for the people/organizations providing them who have to pay for the bandwidth.
Now, the solution seems obvious: leverage the power of peer-to-peer technology to spread the load out among Ubuntu's now vast user base. There are already torrents available for the CD downloads, and these normally work quite well - provided users are aware of them (and everyone did a great job this time around making a point to urge people to use bittorrent when possible, so that's great). However, this doesn't help those who wish to do a network upgrade rather than getting a new CD.
So, what I would like to see implemented is developing a way of integrating P2P technology, preferably the bittorrent protocol, into apt, and making this either extremely easy to enable (and quite obvious to the user that it exists and that they should use it), or perhaps even the default mechanism. Another option would be to have release dates somehow programmed in, and have users' machines switch to the torrent version of operation around release day, but use http the rest of the time.
There are two tools I am aware of that have started trying to address this issue. I have heard that the latter is a more advanced implementation of the solution, but I'm not sure.
See also:
http://sianka.free.fr/ - Apt-Torrent
http://debtorrent.alioth.debian.org/ - DebTorrent
Delicious
Digg
StumbleUpon
Reddit
Facebook
Google
Technorati
Comments
I thought about this very
I thought about this very idea when I tried updating on release date. I was able to grab the ISOs without any problem. With bittorrent you more who are downloading, the more availability. With the traditional method, the more people downloading, the less availability.
Integrating bittorent technology with apt will make release days a pleasure as more people start downloading, and thus, start seeding.
I'm completely biased (being
I'm completely biased (being the DebTorrent developer), but in my opinion DebTorrent IS a more advanced implementation. apt-torrent is basically an http server for communicating with APT, and a wrapper around bittornado to do the actual downloading. DebTorrent implements the http server together with a bittorrent-type client. The modifications to the standard bittorrent protocol allow DebTorrent to be more efficient and complete, combining packages into larger torrents (so you don't have to run too many), and supporting all packages in the archive (whereas apt-torrent currently only supports the 100 largest). I added an FAQ entry about the difference between them here:
http://debtorrent.alioth.debian.org/FAQ.html#index4h4
DebTorrent should be available in Ubuntu very soon, and I hope will be used by some for the upgrade to Hardy (probably not too many though, since it didn't make it into Gutsy and so won't be available to most people at upgrade time). Once in Hardy though it should make the upgrade to Hardy+1 more manageable for users and the archive admins.
Another option would be
Another option would be http://metalinker.org/ which incorporates torrent, as well as ftp/http multi-segment-downloading. That'd also be perfect for .iso distribution.
Fantastic idea. I think the
Fantastic idea. I think the torrent for all installed packages should be enabled by default (with a default throttle to keep it from killing people's connections), and just make it very obvious and easy to disable. I think most geeks with broadband would be happy to seed packages most of the time. As for the "business case", I don't see how it could be a bad thing to promote torrents as legitimate traffic, and it would be easy enough to disable for companies that are too conservative to give it a try.
One of the things about
One of the things about bittorrent is that it works best with big files that everyone is downloading at the same time.
Wouldn't it make sense if you could (optionally) download the alternate .iso, have it mounted in some manner, and then let the standard apt tools access the files found there.
This means you would be downloading the same image as the disc burners.
As an added bonus a single machine could download this image and then make it available to nearby machines via zeroconf.
a) .debs have to be kept on
a) .debs have to be kept on the computer, which is around ~700MB more disk space on average ubuntu installations
b) most users do not want to upload permanently and feel unsafe. also, their internet speed may go down
c) p2p is blocked by some isps in the us, and is disabled in lots of companies' networks
-> most people would not do that (as it needs to be optional and has some shortcomings) so there would not be a big speed increase. a better solution would be to introduce a good load balancing infrastructure.
also, there is a blueprint available at https://blueprints.launchpad.net/ubuntu/+spec/apt-torrent which was not assigned to a sprint which i did now. that may be help more than just pushing features that you would like to see in your blog.
what if everyone on planet ubuntu would push his wishlist items? there are 1651 blueprint items against ubuntu..
take care about the
take care about the opportunity to use always http protocol 'cause in most "businness" case there isn't a direct access to internet but just via proxy or some firewall rules,
and very often torrent protocol is forbidden! so if u want ubuntu to be used in a professiona environment you need to allow this
well, isn't that why conary
well, isn't that why conary came in ?
Debian already use
Debian already use this
Maybe its time to gather commetn from debianist
Great idea ! It would
Great idea ! It would definitely cut the load on the repositories servers. (The french one are down since Gutsy release...)
Excellent idea! Downloading
Excellent idea! Downloading the Gutsy alternate CD image using bittorrent was very convenient, but the upgrades from the repositories were a pain. I hope to see APT using bittorrent in Hardy.
There are other issues, than
There are other issues, than the whole "p2p is blocked" and "I don't want to have to seed debs and store them on my system for a while."
There was a case recently where someone wrote a proof-of-concept using a ps3 to on the fly re-hash a bitorrent part after swapping the contents so that the hash meets the check but contains a binary that is not what should be there...
So #apt-torrent install ssh-server
That then has a compromised ssh server that now has root privileges and is one awesome rootkit... I guess if each part was gpg-signed it might be safe for the time being.
Other than those three issues, I love the idea. :)
Should be safe
I was not aware of the proof of concept you mention (although it certainly sounds interesting), but you'll be pleased to know that every package in the Ubuntu repositories (including mirrors) is individually GPG signed, and apt will spit out a fairly explicit warning if you try to install one that either a) is not signed, or b) is signed with a signature not registered with APT as a trusted source. Granted, many users ignore warnings, but there are specs very heavily under way to make the security issues regarding packages to install much clearer.