User avatar
Gavinmc42
Posts: 3935
Joined: Wed Aug 28, 2013 3:31 am

emerge, ebuild, binhost

Sat Aug 24, 2019 6:10 am

A beginners guide to Gentoo64 app installing from a Gentoo beginner.
Sakaki can correct any of my mistakes here.
Reading ALL her wiki's is very useful.

If you are new to Gentoo64 you might have noticed Gentoo does not use Debian's/Raspbian's apt-get install.
Gentoo builds applications from source as the default, this can take a very long time.
Some apps have been prebuilt and are located on what Gentoo calls Binhosts.
Explanation - https://www.edoceo.com/howto/gentoo-portage-binhost

For Gentoo64 on Pi4, look here first
https://isshoni.org/pi64pie/

But why build apps from source if it takes so long?
They will be more up to date, hopefully that means less bugs, more performance.
And for us with Pi4's they will be optimised for ARMv8 A72 thanks to Sakaki's configs

Some Pi4 speeds test are showing about a 10-15% increase in speed over 32bit Buster.
When Pi4's are so close to the bottom edge in Desktop Performance every bit of speed counts :D

So if an app is built on one Pi4, how to use it on a second Pi4 without rebuilding from source again.
I have no idea yet :lol:
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

User avatar
Gavinmc42
Posts: 3935
Joined: Wed Aug 28, 2013 3:31 am

Re: emerge, ebuild, binhost

Sat Aug 24, 2019 7:39 am

If you have a Pi4B2, you will run out of memory to build Kicad if you have Chromium open with about 8 tabs. :oops:

For those that have serious PC's
https://github.com/sakaki-/gentoo-on-rp ... h-crossdev

As Sakaki points out in the main post this is what is used for the big compiles like Rust.
https://www.packet.com/cloud/servers/c1-large-arm/

How big an app can a headless Gentoo64 Pi4B4 build?
How long will it take?
Or a cluster with 4 x Pi4B4, that's 16 Arm cores and 16GB of RAM.
Learning these answers could be fun.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

User avatar
Gavinmc42
Posts: 3935
Joined: Wed Aug 28, 2013 3:31 am

Re: emerge, ebuild, binhost

Sat Aug 24, 2019 9:16 am

Back to those built binaries
https://forums.gentoo.org/viewtopic-t-1 ... art-0.html

Look here and you will find packages.
/usr/portage/packages/
It looks like the ones that comes with Gentoo64 and the ones you build.
I suppose the Pi you build on just needs a simple file server so your other Gentoo64 Pi's can install them without building.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

geev03
Posts: 131
Joined: Thu Jun 07, 2012 12:40 pm
Location: London, UK

Re: emerge, ebuild, binhost

Sat Aug 24, 2019 11:59 am

'docker has emerged'..

Code: Select all

geev03@pi64 ~ $ sudo -i
pi64 ~ # docker run -i -t armhf/alpine
/ # apk update
fetch http://dl-cdn.alpinelinux.org/alpine/v3.5/main/armhf/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.5/community/armhf/APKINDEX.tar.gz
v3.5.3-45-g195c4dea45 [http://dl-cdn.alpinelinux.org/alpine/v3.5/main]
v3.5.3-44-g08ec10f083 [http://dl-cdn.alpinelinux.org/alpine/v3.5/community]
OK: 7846 distinct packages available
/ # apk upgrade
Upgrading critical system libraries and apk-tools:
(1/1) Upgrading apk-tools (2.6.8-r2 -> 2.6.10-r0)
Executing busybox-1.25.1-r0.trigger
Continuing the upgrade transaction with new apk-tools:
(1/4) Upgrading musl (1.1.15-r6 -> 1.1.15-r8)
(2/4) Upgrading busybox (1.25.1-r0 -> 1.25.1-r2)
Executing busybox-1.25.1-r2.post-upgrade
(3/4) Upgrading zlib (1.2.8-r2 -> 1.2.11-r0)
(4/4) Upgrading musl-utils (1.1.15-r6 -> 1.1.15-r8)
Executing busybox-1.25.1-r2.trigger
OK: 3 MiB in 11 packages

User avatar
sakaki
Posts: 381
Joined: Sun Jul 16, 2017 1:11 pm

Re: emerge, ebuild, binhost

Sat Aug 24, 2019 4:40 pm

Gavinmc42 wrote:
Sat Aug 24, 2019 9:16 am
Back to those built binaries
https://forums.gentoo.org/viewtopic-t-1 ... art-0.html

Look here and you will find packages.
/usr/portage/packages/
It looks like the ones that comes with Gentoo64 and the ones you build.
I suppose the Pi you build on just needs a simple file server so your other Gentoo64 Pi's can install them without building.
Yes, have a look at the following for an intro to creating, managing and distributing binary packages:

https://wiki.gentoo.org/wiki/Binary_package_guide

The gentoo-on-rpi-64bit image is set up to create a binary package automatically each time you emerge something, as a backup and to allow (as you point out) other 64-bit Pis to use the package without installation.

As to compilation efficiency, make sure (if you have a >=2GiB unit) to at least uncomment the MAKEOPTS and EMERGE_DEFAULT_OPTS in /etc/portage/make.conf:

Code: Select all

# override default build parallelism by uncommenting the following
# two lines, and modifying the values as required
# Pi4 users (particularly those with a 4GiB variant) can afford to be
# more aggressive here
#MAKEOPTS="-j5 -l4"
#EMERGE_DEFAULT_OPTS="--jobs=5 --load-average=4"
# caution! many parallel threads and / or emerge jobs may cause your
# RPi3/4 to run out of memory during large builds, or genup runs
as by default I have make parallelism clamped very low (to prevent issues for e.g. users on Pi3Bs).

best, sakaki

User avatar
Gavinmc42
Posts: 3935
Joined: Wed Aug 28, 2013 3:31 am

Re: emerge, ebuild, binhost

Sun Aug 25, 2019 12:39 am

Thanks Sakaki, I think you have convinced me to wait for a 4B4 and use that as a headless build box.
Big Builds and browsing on a 2GB is not optimal.
I did notice you had the compiler option set at -j2 ;)

So far my experience with Gentoo64 is,
Pi4B1 is ok for browsing and normal desktop computing.
Pi4B2 is ok for browsing, computing and small ebuilds.
If the 2GB was headless perhaps bigger builds?
Pi4B4 should be ok for big builds.
A cluster of Pi4B4's could do ?

How do I use those builds on your Binhost?
Eg Openscenegraph, I can download but how to install?
Can I push my builds onto it? Geany, OpenSCAD, WarZone2100...
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

User avatar
sakaki
Posts: 381
Joined: Sun Jul 16, 2017 1:11 pm

Re: emerge, ebuild, binhost

Sun Aug 25, 2019 9:59 am

Gavinmc42 wrote:
Sun Aug 25, 2019 12:39 am
How do I use those builds on your Binhost?
Eg Openscenegraph, I can download but how to install?
Can I push my builds onto it? Geany, OpenSCAD, WarZone2100...
So, if a tbz2 file (binary package) exists on the binhost, and you try to emerge it locally, then given the way things are set up it should automatically offer to pull the file down from the binhost and install it. Try e.g.:

Code: Select all

demouser@pi64 ~ $ sudo emerge -av dev-games/openscenegraph
and you should see something like the following output:

Code: Select all

Local copy of remote index is up-to-date and will be used.

These are the packages that would be merged, in order:

Calculating dependencies... done!
[binary  N    *] dev-games/openscenegraph-3.6.3:0/158::gentoo  USE="egl ffmpeg gif gtk jpeg pdf png sdl svg tiff truetype wxwidgets zlib -asio -curl -debug -doc -examples -fltk -fox -gdal -glut -gstreamer (-jpeg2k) -las -libav -lua -openexr -openinventor -osgapps -sdl2 -vnc -xrandr" 9,932 KiB

Total: 1 package (1 new, 1 binary), Size of downloads: 9,932 KiB

Would you like to merge these packages? [Yes/No]
You can see this is going to use a binary package, rather than an ebuild, and the download size of the tbz2 is (here) 9,932 KiB.

If e.g. your local USE flags don't match with those used when the binhost package was built, Portage will warn you that a binary package is being ignored. So, for example, if you were to force on the lua USE flag (NB: don't set flags this way in real use, instead set them via an entry in /etc/portage/packages.use; this is just a quick example):

Code: Select all

demouser@pi64 ~ $ sudo USE=lua emerge -av openscenegraph

Local copy of remote index is up-to-date and will be used.

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N    *] dev-games/openscenegraph-3.6.3:0/158::gentoo  USE="egl ffmpeg gif gtk jpeg lua pdf png sdl svg tiff truetype wxwidgets zlib -asio -curl -debug -doc -examples -fltk -fox -gdal -glut -gstreamer (-jpeg2k) -las -libav -openexr -openinventor -osgapps -sdl2 -vnc -xrandr" 5,650 KiB

Total: 1 package (1 new), Size of downloads: 5,650 KiB

!!! The following binary packages have been ignored due to non matching USE:

    =dev-games/openscenegraph-3.6.3 -lua

NOTE: The --binpkg-respect-use=n option will prevent emerge
      from ignoring these binary packages if possible.
      Using --binpkg-respect-use=y will silence this warning.

Would you like to merge these packages? [Yes/No]
As prompted above, you can force the use of the binhost package regardless of USE flags by specifying --binpkg-respect-use=n (this is sometimes useful when working with a new package and you just want something to run to get started - but is not recommended generally as you can get into dependency mismatches very quickly). For example:

Code: Select all

demouser@pi64 ~ $ sudo USE=lua emerge -av --binpkg-respect-use=n openscenegraph

Local copy of remote index is up-to-date and will be used.

These are the packages that would be merged, in order:

Calculating dependencies... done!
[binary  N    *] dev-games/openscenegraph-3.6.3:0/158::gentoo  USE="egl ffmpeg gif gtk jpeg pdf png sdl svg tiff truetype wxwidgets zlib -asio -curl -debug -doc -examples -fltk -fox -gdal -glut -gstreamer (-jpeg2k) -las -libav -lua -openexr -openinventor -osgapps -sdl2 -vnc -xrandr" 9,932 KiB

Total: 1 package (1 new, 1 binary), Size of downloads: 9,932 KiB

Would you like to merge these packages? [Yes/No]
The remote package index is updated on each emerge, so you don't have to do anything to keep it up to date. This is why on genup runs, where possible binary packages are automagically used. Note the system will download missing tbz2s from the binhost into /usr/portage/packages, where they will co-mingle with any tbz2s you have created locally.

There's no easy way to add your own packages to the current isshoni.org binhost as I have a pretty strict version control system set up between this and my staging test box. However, if you get a package to ebuild, you can request I add this to the build list, which will then maintain its tbz2s going forward.

hope that makes sense!

Best, sakaki

User avatar
Gavinmc42
Posts: 3935
Joined: Wed Aug 28, 2013 3:31 am

Re: emerge, ebuild, binhost

Sun Aug 25, 2019 10:49 am

Thanks Sakaki , makes perfect sense :D
And it installs faster than debs.
One dependency crossed off the Flightgear list.

Don't think I have noticed that binary emerge option before.

OpenSCAD was an ebuild.
Kicad ebuild is a big one that broke my Pi4B2 ;)
Inkscape and Blender would be nice.
FreeCAD should be doable as some of the dependencies built.
Arduino IDE has an aarch64 version that worked, but is probably not A72 optimised.

Those apps turn an office machine into a designers desktop.

Not sure about my other stuff, they have been compiled from source not ebuilt.
Most of these others are esoteric, not mainstream.

Yep, I did not think you wanted others adding stuff to the binhost.
Considering most of the time I trying things and just hoping they work using that "n" option.
What's version control?
Most of my gadgets are one off's, trying to reuse code and methods has driven me and the boss nuts for years.

Thanks again, a little bit more of the Gentoo world has been explained.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

User avatar
Gavinmc42
Posts: 3935
Joined: Wed Aug 28, 2013 3:31 am

Re: emerge, ebuild, binhost

Mon Aug 26, 2019 12:18 am

And about 1/2 hour later I had to emerge subversion to try getting free pascal installed :lol:
So an ebuild of dev-vcs/subversion would be nice.

This can be an issue with software that have their own install methods.
Can ebuilds be made of these?
How can compiled binaries be made for Gentoo64 to go on Binhosts?
More learning :lol:
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

User avatar
Gavinmc42
Posts: 3935
Joined: Wed Aug 28, 2013 3:31 am

Re: emerge, ebuild, binhost

Sun Sep 29, 2019 9:08 am

Broke my big version 64GB uSD Gentoo64.
Did a genup and reboot and it did not boot :(

Bit the uSD was still readable from a Pi3 with USB boot Gentoo64.
So all my built packages and emerge were there.
Is there a way to reinstall them from the packages and not rebuilding from source again?

Going to need to know this if I keep breaking my Gentoo64.
Time to set up my own BINHOST now.

I don't suppose you know of a Gentoo NAS spin?
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

User avatar
sakaki
Posts: 381
Joined: Sun Jul 16, 2017 1:11 pm

Re: emerge, ebuild, binhost

Sun Sep 29, 2019 12:42 pm

Gavinmc42 wrote:
Sun Sep 29, 2019 9:08 am
Broke my big version 64GB uSD Gentoo64.
Did a genup and reboot and it did not boot :(

Bit the uSD was still readable from a Pi3 with USB boot Gentoo64.
So all my built packages and emerge were there.
Is there a way to reinstall them from the packages and not rebuilding from source again?
You can always copy /usr/portage/packages across onto a new image; the contents (i.e. the binary packages for anything you've emerged locally plus anything emerged from a binary off the isshoni.org binhost) will then be used during emerges where possible (since the getbinpkg FEATURE is set).

You should try to fix your uSD image first though. Does it boot at all (i.e., does the kernel start up)? I have only just (i.e., today) unlocked a change to the kernel wrt the 1.5.1 release, so unless you have been unmasking new kernels locally, nothing should really have changed /boot...

Best, sakaki

User avatar
Gavinmc42
Posts: 3935
Joined: Wed Aug 28, 2013 3:31 am

Re: emerge, ebuild, binhost

Mon Sep 30, 2019 1:18 am

Thanks Sakaki,

Been emerging crossdev stuff and genup.
I'm not surprised, still mostly in the dark and bumping into walls with Gentoo.

Fixed my broken Buster uSD, config.txt issue after an update/upgrade.
Now suspect a similar issue with Gentoo64.
Blinking green led is not blinking normally.

More important now is a Gentoo64 Backup system using spare Pi's.
I fully expect to break Pi OS's again.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

User avatar
Gavinmc42
Posts: 3935
Joined: Wed Aug 28, 2013 3:31 am

Re: emerge, ebuild, binhost

Sat Nov 02, 2019 1:20 am

What to do if what you want to emerge is "masked".

Copying this here as I fully expect to lose it again.
Ok, found the wiki and Gentoo Handbook.
https://github.com/sakaki-/gentoo-on-rp ... der-gentoo
Sakaki writes great wiki's, but if you don't read them you cannot find out how to do things :oops:
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

geev03
Posts: 131
Joined: Thu Jun 07, 2012 12:40 pm
Location: London, UK

Re: emerge, ebuild, binhost

Sat Nov 02, 2019 7:14 pm

Gavinmc42 wrote:
Sat Nov 02, 2019 1:20 am
What to do if what you want to emerge is "masked".

Copying this here as I fully expect to lose it again.
Ok, found the wiki and Gentoo Handbook.
https://github.com/sakaki-/gentoo-on-rp ... der-gentoo
Sakaki writes great wiki's, but if you don't read them you cannot find out how to do things :oops:
I read the wiki . :D .. as can be seen by the attached file showing ' sudo emerge rasmol'
It was a good learning exercise , included insertion of the following (and more) codes as needed. :mrgreen:

Code: Select all

echo "sci-chemistry/rasmol * ~*" >> /etc/portage/package.accept_keywords/rasmol
echo "dev-libs/cvector * ~*" >> /etc/portage/package.accept_keywords/cvector
echo "dev-util/gtk-builder-convert * ~*" >> /etc/portage/package.accept_keywords/gtk-builder-convert
echo " sci-libs/cbflib * ~*" >> /etc/portage/package.accept_keywords/cbflib
Attachments
gentoo_rasmol.jpg
gentoo_rasmol.jpg (139.72 KiB) Viewed 1115 times

User avatar
Gavinmc42
Posts: 3935
Joined: Wed Aug 28, 2013 3:31 am

Re: emerge, ebuild, binhost

Sun Nov 03, 2019 1:21 am

Emerge is like the Debian/Raspbian deb packaging method which I have not learned or bothered to learn.
While more debs may exist some are outdated or like Lazarus install it so it is half usable.

I am coming to appreciate the Gentoo method of building from source even if it is slower.
You end up with a more personalised OS, but it is not for beginners without Sakaki's images, help and wikis..

Getting tired of emerging everything every time and it's now time to make a binhost and backup server.
The P4B4 is spoiling me, emerging/compiling and browsing or playing games at the same time :D

Last night crossdev installed a 32bit compiler

Code: Select all

demouser@pi64 ~ $ gcc-config --list-profiles
 [1] aarch64-unknown-linux-gnu-9.2.0 *

 [2] armv6j-hardfloat-linux-gnueabi-8.3.0 *
demouser@pi64 ~ $ 
Not a lot of instructions for crossdev on arm64 but guess who's instructions worked ;)
Took me 4-5 failed attempts to install crossdev tools following other old instructions.
Coming to a grinding halt as my 64GB uSD is nearly full.

Some way to package and run the application from a USB stick would be nice.
By having the app and working files stored on a USB stick swapping projects is just a USB swap.
Not sure if this is the answer https://flatpak.org/
Perhaps something like this
https://portableapps.com/
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

User avatar
Gavinmc42
Posts: 3935
Joined: Wed Aug 28, 2013 3:31 am

Re: emerge, ebuild, binhost

Mon Nov 04, 2019 8:16 am

I have emerged quite a few apps many times now as I tend to break my Gentoo64 uSDs.
Been looking at flatpak and appimage etc.
I don't have home server so a personal BINHOST is still some time away.

Is there a way to put my binary ebuilds on a USB stick and install them from there?
That way I can keep backups of binary ebuilds and reload them when I reimage a new Gentoo.

Ideally running them from the USB stick would be nice.
OS on the uSD, apps on the USB.
It also save lots of uSD card expense as I can use smaller sizes.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

User avatar
Gavinmc42
Posts: 3935
Joined: Wed Aug 28, 2013 3:31 am

Re: emerge, ebuild, binhost

Sat Nov 16, 2019 5:33 am

In the PC x86 world there is Portable apps, I have used a few before and they just work.
https://portableapps.com/about

As Gentoo already makes ebuilds which are basically zipped installs, can they be unzipped from a USB stick to a ram drive and run from there?
Pi4 with 4GB seems to have enough to spare most of the time.
PiCore Linux sort of does this run from ram method.

A Pi4 with a USB SSD is a bit clunky to carry around, plus USB3 to Sata don't seem stable yet.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

Return to “Gentoo”