The over-temperature icon is flickering on and off continuously on the Pi4/4 I'm running as a second desktop, alongside my 'proper' desktop. It's fitted with heatsinks and mounted in a modified old-type official case with both side panels and the top cover left off. On current Buster it's running one program – the Chrome browser with two tabs open, one of them displaying a national newspaper and the other streaming BBC iPlayer Radio. It's not an especially warm day.
By and large I'm quite impressed with this new version, but I think we're kidding ourselves if we think it's suitable for use as a desktop without a cooling fan.
Edit: I note that the new Pi4 case, if you can get one, doesn't have the removable side panels and top cover, which will presumably make matters worse.