apt update issues

[MarkJ62]

A Pi3B+ running Buster. No proxy. Taking 9 minutes to get the list of updates and its 69MB. I have repeated this on a couple of Pi's in my bramble. They're all configured the same. Does anyone get three different sizes for their Packages or is it just me?

$ uname -a
Linux *** 4.19.58-v7+ #1245 SMP Fri Jul 12 17:25:51 BST 2019 armv7l GNU/Linux

$ sudo apt --version
apt 1.8.2 (armhf)

$ sudo apt update
Get:1 http://archive.raspberrypi.org/debian buster InRelease [25.1 kB]
Get:2 http://raspbian.raspberrypi.org/raspbian buster InRelease [15.0 kB]
Get:3 http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages [13.0 MB]
Get:4 http://archive.raspberrypi.org/debian buster/main armhf Packages [222 kB]
Ign:3 http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages
Get:3 http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages [18.3 MB]
Ign:3 http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages
Get:3 http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages [69.0 MB]
Fetched 69.3 MB in 9min 9s (126 kB/s)
Reading package lists... Done

Packages appears to gone from 13MB to 18.3MB and then to 69MB. Are there 3 sets of Packages?

$ cat /etc/apt/sources.list
deb http://raspbian.raspberrypi.org/raspbian/ buster main contrib non-free rpi
# Uncomment line below then 'apt-get update' to enable 'apt-get source'
#deb-src http://raspbian.raspberrypi.org/raspbian/ buster main contrib non-free rpi

$ cat /etc/apt/sources.list.d/raspi.list
deb http://archive.raspberrypi.org/debian/ buster main
# Uncomment line below then 'apt-get update' to enable 'apt-get source'
#deb-src http://archive.raspberrypi.org/debian/ buster main

[rpdom]

Packages appears to gone from 13MB to 18.3MB and then to 69MB. Are there 3 sets of Packages?

No, there are five sets of packages files in your set up. But two of them haven't changed since last time you did an apt update, so they didn't need to be downloaded again "Ign:"

For this line in your sources.list files there are four sets of packages file

Code: Select all

deb http://raspbian.raspberrypi.org/raspbian/ buster main contrib non-free rpi

All from http://raspbian.raspberrypi.org/raspbian/ for the buster release. One for "main", one for "contrib", one for "non-free" and one for "rpi".

Then you have one more for "main" in the archive.raspberrypi.org raspbian area.

[Milliways]

A Pi3B+ running Buster. No proxy. Taking 9 minutes to get the list of updates and its 69MB. I have repeated this on a couple of Pi's in my bramble. They're all configured the same. Does anyone get three different sizes for their Packages or is it just me?

Since the release of Buster update takes ages (even Stretch) and more often than not fails.

I have changed to a local mirror chosen from https://www.raspbian.org/RaspbianMirrors/

Code: Select all

# deb http://raspbian.raspberrypi.org/raspbian/ buster main contrib non-free rpi
deb http://raspbian.melbourneitmirror.net/raspbian/ buster main contrib non-free rpi

[MarkJ62]

Packages appears to gone from 13MB to 18.3MB and then to 69MB. Are there 3 sets of Packages?

No, there are five sets of packages files in your set up. But two of them haven't changed since last time you did an apt update, so they didn't need to be downloaded again "Ign:"

For this line in your sources.list files there are four sets of packages file

Code: Select all

deb http://raspbian.raspberrypi.org/raspbian/ buster main contrib non-free rpi

All from http://raspbian.raspberrypi.org/raspbian/ for the buster release. One for "main", one for "contrib", one for "non-free" and one for "rpi".

Then you have one more for "main" in the archive.raspberrypi.org raspbian area.

Can I refer you to this message in the Buster bug reports thread.

In that thread I was using squid to try and cache stuff for the Bramble. It would appear that it gets a Packages.gz, and then 3 more Packages.xz.that seem to originate from the same url and have the same names. I presume these correlate with the main, contrib, non-free and rpi parts of the repo. If I am correct then squid would be unable to cache the Packages.xz because they appear to be the same file but the attributes are different (sizes and/or dates). That might also account for the load on the server.

[rpdom]

But the thing is they don't have the same names.

Yes, they are all "Packages.xz" and they are all from the same "http:////raspbian.raspberrypi.org/" server, but the path names of each file are different. If your proxy/cache is considering them to be the same it has a serious bug in it.

This shows the files that would be looked at and downloaded if neccessary

Code: Select all

pi@raspi12:~ $ apt-get --print-uris update
'http://archive.raspberrypi.org/debian/dists/buster/InRelease' archive.raspberrypi.org_debian_dists_buster_InRelease 0 
'http://archive.raspberrypi.org/debian/dists/buster/main/binary-armhf/Packages.xz' archive.raspberrypi.org_debian_dists_buster_main_binary-armhf_Packages 0 
'http://archive.raspberrypi.org/debian/dists/buster/main/binary-all/Packages.xz' archive.raspberrypi.org_debian_dists_buster_main_binary-all_Packages 0 
'http://raspbian.raspberrypi.org/raspbian/dists/buster/InRelease' raspbian.raspberrypi.org_raspbian_dists_buster_InRelease 0 
'http://raspbian.raspberrypi.org/raspbian/dists/buster/main/binary-armhf/Packages.xz' raspbian.raspberrypi.org_raspbian_dists_buster_main_binary-armhf_Packages 0 
'http://raspbian.raspberrypi.org/raspbian/dists/buster/main/binary-all/Packages.xz' raspbian.raspberrypi.org_raspbian_dists_buster_main_binary-all_Packages 0 
'http://raspbian.raspberrypi.org/raspbian/dists/buster/contrib/binary-armhf/Packages.xz' raspbian.raspberrypi.org_raspbian_dists_buster_contrib_binary-armhf_Packages 0 
'http://raspbian.raspberrypi.org/raspbian/dists/buster/contrib/binary-all/Packages.xz' raspbian.raspberrypi.org_raspbian_dists_buster_contrib_binary-all_Packages 0 
'http://raspbian.raspberrypi.org/raspbian/dists/buster/non-free/binary-armhf/Packages.xz' raspbian.raspberrypi.org_raspbian_dists_buster_non-free_binary-armhf_Packages 0 
'http://raspbian.raspberrypi.org/raspbian/dists/buster/non-free/binary-all/Packages.xz' raspbian.raspberrypi.org_raspbian_dists_buster_non-free_binary-all_Packages 0 
'http://raspbian.raspberrypi.org/raspbian/dists/buster/rpi/binary-armhf/Packages.xz' raspbian.raspberrypi.org_raspbian_dists_buster_rpi_binary-armhf_Packages 0 
'http://raspbian.raspberrypi.org/raspbian/dists/buster/rpi/binary-all/Packages.xz' raspbian.raspberrypi.org_raspbian_dists_buster_rpi_binary-all_Packages 0 

The first part of each line is the file to fetch.
The second part is what the file will be called when saved locally.