[GRLUG] One HUGE software repository

[Greg Folkert]

On Tue, 2008-02-26 at 22:45 -0500, George (Skip) VerDuin wrote:
> It looks like I made my next mistake yesterday.  The debian mirror is an
> awesome thing.  Yet I'm far enough into it that I hate to cut bait
> now...
> 
> I mentioned earlier that when I rebuilt my local system repeatedly for
> practice it looked to me like having a local mirror would save wear and
> tear on my DSL link and not irritate the debian guys.  The systems I
> built took a couple hrs for the basic system and a couple more for the
> desktop build.  I am now at 25GB and in the "i"s headed toward "z" after
> a day [16(+)hrs] connected by rsync to the MSU repository.  And that
> statistic is based on a long laundry list of exclusions from the full
> set of materials.  This moment is not a full 24hr stint yet because the
> transfer fell apart some time last night for unknown reasons.
> 
> I'm not certain where I went wrong yet, but after reading repository
> documentation I expected maybe DSL hrs of 4X times a full desktop system
> load to have the library local.  It may be more like 12X.
> 
> But quite frankly I probably would consume fewer DSL hrs building from a
> Internet repository VS bringing it here.  *Not* maybe well done by yours
> truly.
> 
> For anyone else in the group considering similar madness -- if you want
> to share in it I can now claim to now have some experience?

Uhhh wow.

You have to remember, Debian has 13 platforms for which it has packages.
PLUS sources.

You have to exclude packages and other things for: PA-RISC, SPARC, PPC,
IA64 (Itanium), SH, OS390, Alpha, ARM, MIPS, MIPS-EL

Otherwise the entire repository is about 90+GB.

I used to host a Partial Debian repository. I had a TON OF EXCLUSION
PATTERNS (like 70-80) for my rsync script. BTW, rsync is the preferred
way to go for mirroring a Debian repository.

The initial D/L took 7 days and then 2 days to do the incremental, then
another day to finally catch up. Of course I was on a 1.1MiB line at the
time, I was maintaining 30 or so machines.

Now, to me its not even worth it any more.
-- 
greg at gregfolkert.net
PGP key 1024D/B524687C 2003-08-05
Fingerprint: E1D3 E3D7 5850 957E FED0  2B3A ED66 6971 B524 687C
Alternate Fingerprint: 09F9 1102 9D74  E35B D841 56C5 6356 88C0
Alternate Fingerprint: 455F E104 22CA  29C4 933F 9505 2B79 2AB2
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://shinobu.grlug.org/pipermail/grlug/attachments/20080227/648b4cab/attachment-0001.pgp