why is hardware random number generator not set up by default?

[skypi]

why is hardware random number generator not set up by default?

seems to me, it makes entropy at boot time like 2000 whatevers better than a standard installation...

The standard installation eventually reaches similar 3300 whatevers entropy after a while, as the hwrng enabled configuration, but is there even still a subtle improvement with hwrng enabled? (seems about 100 whatevers better all the time after long uptime)

cat /proc/sys/kernel/random/entropy_avail

thing is, ssh keys are generated at first boot, I'm not sure but without hwrng enabled isnt it very low entropy at that point?

to me does not make sense it is not enabled, as it is standard hardware across all models.

is there a good reason? (gchq infiltrators?)

[DougieLawson]

It is set up. /dev/hwrng exists on all 14 of my Raspberries (13 run a 4.14.15 kernel, one runs a 4.9.60 kernel) all are running Stretch.

[skypi]

strange, just installed a stretch installation and checked and rng-tools.service is not running without manually installing, and after reboot....... let me just do that now to check... two pi zeros, one with rng installed one without

without rng tools:

uptime
20:27:17 up 0 min, 1 user, load average: 1.49, 0.45, 0.16
cat /proc/sys/kernel/random/entropy_avail
883

with rng tools:

uptime
20:27:10 up 0 min, 1 user, load average: 0.98, 0.29, 0.10

cat /proc/sys/kernel/random/entropy_avail
2116


so maybe loading the rngtools service excercises the hwrng, but standard installation catches up after a while ???? (and gchq infiltrators)

it's a bit apples and oranges the one with rng tools also has camera enabled, whereas without does not....

[jahboater]

It is set up. /dev/hwrng exists on all 14 of my Raspberries (13 run a 4.14.15 kernel, one runs a 4.9.60 kernel) all are running Stretch.

It needs root to read it, which is a bit annoying.
Compare it to Intel x86, where you have the rdrand and rdseed instructions available to all processes, no special privilege required.

[skypi]

right!, thanks, so that's why this did not work

dd if=/dev/hwrng count=1 2>/dev/null | base64 | head -1 | cut -c4-15

should be

sudo dd if=/dev/hwrng count=1 2>/dev/null | base64 | head -1 | cut -c4-15

[Heater]

I would not want to use rdrand instructions.

Those who are into cryptography don't trust them: https://en.wikipedia.org/wiki/RdRand Better to use /dev/random and /dev/urandom.

For other uses there are very fast and high quality pseudo random number generator algorithms you can use. For example: http://xoroshiro.di.unimi.it/

With that added advantage that if you use those your code is portable and the results reproducible if need be.

[skypi]

so? does entropy_avail reflect getting a number direct from hwrng? if not what is relative entropy using same scale from hwrng????

but is rdrand same from arm as from intel? (gchq infiltration?), I mean if they can infiltrate the cpu manufacturers cryptography, sure as heck they got plenty of infiltrators on the open-source crews eh!

good article on the subject

https://www.theregister.co.uk/2013/09/1 ... _nsa_gchq/

(like what probability of snowden being ultimate backdoor???? paranoia strikes deep! into your minds it will creep! it starts when you're always afraid!)

anyway, I need to do an apples and apples comparison, two stock images on two absolute same-spec pi's, only change rng-tools installed or not, and seewhat happens to entropy avail over time... (in repeated tests) say you know at T(n) one will be low entropy vs the other, having noticed one without rng-tools caught up and exceeded one with rng-tools, still a way of weakening the armour given knowlege of T(n) available

[jahboater]

Heater,
rdrand and rdseed conform to various cryptographic standards: NIST SP800-90A, B, and C, FIPS-140-2, and ANSI X9.82. I know some people think the NSA have a back door, but who knows? I doubt it. I don't think those xorshift generators in the link are in the same league by the way - certainly not cryptographically secure. Even MT19937 with its huge period is not.

rdrand (but not rdseed which is I believe a TRNG) can produce tens of GB/sec of numbers. Much faster than opening a file, calling read() and so on.

[Heater]

jahboater,

I know some people think the NSA have a back door, but who knows? I doubt it.

I don't know for sure of course. But doubt is enough to keep one away if cryptographic security is what one wants:

https://www.tripwire.com/state-of-secur ... -concerns/

https://arstechnica.com/information-tec ... al-primer/

Regardless of all that, if cryptographic strength is your goal then you should not be trusting a black box.

I don't think those xorshift generators in the link are in the same league by the way - certainly not cryptographically secure. Even the MT19937 with its huge period is not.

It is true that they are not cryptographically secure. That is not their intention and they don't claim to be.

For many applications that is not a requirement and they are plenty random enough. It can be an advantage to have such a PRNG in that results can be reproduced if need be. And it's portable to different machines and operating systems.

rdrand (but not rdseed which is I believe a TRNG) can produce tens of GB/sec of numbers. Much faster than opening a file, calling read() and so on.

True enough.

Now how many percent of the run time of your overall application does that actually save? My guess is that it is very little.

[DougieLawson]

There's lots of still useful words from Stewart Russell (Scruss on here) at: http://scruss.com/blog/2013/06/07/well- ... generator/

We can probably use a udev rule to change permissions, I've tested that with a chmod 666.

[jahboater]

Now how many percent of the run time of your overall application does that actually save? My guess is that it is very little.

I don't know. Monte-carlo simulations need it, but they hardly need the sort of quality that rdrand returns, a well chosen LCG would probably do. Then there are large servers with a lot of encrypted traffic perhaps.

Big subject!

There is a new Linux system call by the way called getrandom() which is intended to one day replace /dev/(u)random. It doesn't need a file descriptor and has some handy flags to control it - for example it can be truly non-blocking. See "man getrandom".

[scruss]

There's lots of still useful words from Stewart Russell (Scruss on here) at: http://scruss.com/blog/2013/06/07/well- ... generator/

aww shucks … I'll still admit to knowing almost nothing about random number generators and security issues. It's something about the impossibility of categorically proving the absence of patterns, I think. It may not be the greatest idea to use /dev/hwrng as your sole source of entropy for a program either, as hardware devices often need a bit of post-processing to remove small biases in their output. That might be the reason for the root permissions on the device.

The other unknown that I'd like to address in that old blog post is whether to use the rng-tools or rng-tools5 packages. It's not clear to me what each package brings.

[Heater]

jahboater,

Thanks for the heads up on getrandom()

Yes, this is a big subject. With a fascinating history.

Amusing story...

A famous name in the history of random numbers on computers is George Marsaglia. The maths professor that created the diehard statistical tests of randomness. When he published diehard on CD back in the day he included some hundreds of megabytes of random bits that passed the diehard tests. Problem was the bits you got on the CD did not pass the tests.

He had used MS-DOS to copy the file of random bits. But he forgot the the "binary" flag to the copy command so MS-DOS treated it as a text file and inserted a line feed (or was it carriage return?) at what it thought were line endings. Thus making the "random" bytes on the CD very much biased.

It's so easy to get these things wrong!

Not George Marsaglia's fault I think. We can thank Bill Gates there for yet another screw up.

[jahboater]

Not George Marsaglia's fault I think. We can thank Bill Gates there for yet another screw up.

thats really funny, I didn't know that one!

George Marsaglia invented the xorshift generators which perform really well (but sadly zero is not a valid state, unlike lcg's). Another name is Pierre L'Ecuyer and on course Knuth, and Matsumoto and Nishimura who invented the twister which is now part of the c++ standard, std::mersenne_twister_engine which has a period of 2^19937.

[jahboater]

It's something about the impossibility of categorically proving the absence of patterns, I think. It may not be the greatest idea to use /dev/hwrng as your sole source of entropy for a program either, as hardware devices often need a bit of post-processing to remove small biases in their output. That might be the reason for the root permissions on the device.

Yes perhaps. The Intel one does elaborate conditioning and testing before random numbers are available to users.

Perhaps also the ARM one has limited output from the entropy source. Or perhaps its just a silly ARM quirk, they also don't make the hardware clock available, unlike Intel who provide the rdtsc instruction to anyone (and put great effort into making it work properly for users).

[Heater]

Highly recommended is this presentation on PCG random number generators by Melissa O'Neill from Stanford:
https://www.youtube.com/watch?v=45Oet5qjlms&t=8s

The PCG generators have some advantage over Mersenne Twister: Does better at statistical tests of randomness, it's harder to predict their output, you can have a lot of parallel streams (good for parallel execution of simulations etc), much smaller and faster.

There is a lot to read about random number generators, PCG in particular, on Melissa's site:
http://www.pcg-random.org

[jahboater]

New and very interesting, thanks.

[skypi]

could be that it is old advice that it was not at some stage in the past set up, but that has changed as it evolves, stm one major problem with internet, you need to look for latest dated article on a subject.

and the entropy_avail not the same as using the hwrng directly then, maybe a mix of using both in generating a single password may be better, they cannot be sure which pool you generated that bit of the password from.

arch as usual has good article on rng-tools

https://wiki.archlinux.org/index.php/Rng-tools

which shows that the install of rng-tools does fill pool with 2048 bits of entropy on startup, but weird thing is (apart from e before i) other zero catches up and exceeds zero with rng-tools at time T(n) but both wind up about same eventually, only 3500, not full 4096!

so a random selection from a pi with rng-tools enabled and from one without at T(random) derived from 3rd pi might be a good random source?

yeah, the arch article is even sceptical enough to say :

"You can override this setting if you really trust your TRNG. To do this"

[Heater]

The Linux random devices /dev/random, /dev/urandom already use a mix of sources of entropy.

[skypi]

yeah, thanks, I had already gathered that, more the precise nature of the thing is what I am investigating! well, at the logical level, not the bit level LOL

spread spectrum eh!

an arduino with a load of sensors randomly ranked by another source as another source....

reflections on trusting trust eh!

[Heater]

I'm quite partial to this circuit for generating random bits:



http://www.cryogenius.com/hardware/rng/

So simple. And how can they backdoor a transistor?

Just needs connecting to a GPIO pin and bit of software to remove any bias and stuff some bits into the Linux entropy pool on boot up.

[jahboater]

You need all this stuff too for conditioning and testing the entropy
See section 3.2.

https://software.intel.com/sites/defaul ... de_2.0.pdf

[RaTTuS]

right!, thanks, so that's why this did not work

dd if=/dev/hwrng count=1 2>/dev/null | base64 | head -1 | cut -c4-15

should be

sudo dd if=/dev/hwrng count=1 2>/dev/null | base64 | head -1 | cut -c4-15

^ that sudo version does not do what you expect it to

[jahboater]

Also you might want to add the "bs=10" argument to dd, otherwise you are taking lots of entropy for just a few bytes of output, and you can then skip the head bit.

[skypi]

So, seems contrary to advice on here the hwrng is not set up to feed /dev/random unless you install rng-tools. (and /dev/urandom is a pseudo number generator once entropy pool is exhausted)

the arch article on rng-tools suggests testing it is working by this command, and if hwrng is set up it will be instantaneous, otherwise will take a long time.

dd if=/dev/random of=/dev/null bs=1024 count=1 iflag=fullblock

I ran the test on zero stretch-lite with rng-tools and one without

with rng-tools

1+0 records in
1+0 records out
1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0192011 s, 53.3 kB/s

without rng-tools

1+0 records in
1+0 records out
1024 bytes (1.0 kB, 1.0 KiB) copied, 98.4036 s, 0.0 kB/s