Most modern Lisps have some sort of FFI interface
FFI is Foreign Function Interface; like I said, Lisp needs a library written in a different language.
Indeed. However, most "base" bit-twiddling functions already exist in Lisp, and the nybble/btye/word swapping functions already exist in libraries that should be more or less universal. Like I said, bad example.
C (and, by extension, C++) has woeful number handling. Great if you're dealing with machine-storage-sized values, useless otherwise. int, in C, does not mean integer.
Limiting integers to machine word-size keeps integer maths time-determinate and efficient, but C gets used in a lot of cases where it shouldn't. On the other hand, there are very few languages where intdoes mean integer. Python and Lisp are the only two I can think of.
Oh, there's lots of languages with full number stacks.
In terms of efficiency, binary numbers aren't particularly efficient unless you restrict yourself to machine word sizes. Increment and decrement (which one might want to be particularly efficient in programming terms), for example, are O(logN) for the general case of binary representation - other representations exist which are O(1) for increment and decrement. In real terms, these might be slower on existing machines (which are tied to 2s complement binary operations), but there's nothing stopping you having a full number stack which is totally deterministic for the type of operations which are particularly useful in programming, and no worse than a generalised 2s complement binary representation for the rest.
I would argue that "hanging on" to machine word sized integers, and (again machine word sized) floating point representation for everything else, is, in this day and age, particularly archaic. Did the Y2K "bug" (no, not tied to machine words, but related in terms of restricting representable data) teach us nothing? The cumulative errors of floating point? The fact that 0.0 is not equal to 0 (and, in many cases, not even equal to 0.0)?
Here's a fun little story. A good few years ago, I was working for a company who dealt with outsourcing. In this case, we were dealing with a warehousing system - stock control, order handling, the lot. As is usually the case with outsourcing deals, we inherited a system where the binaries didn't match to any set of source we had, documentation was nonexistent, and there were numerous manual procedures that had to be carried out every N days/weeks/months or the whole crumbling heap would fall down around your ears, which latter we were strictly unaware of, of course. I, for my sins, was the one who had to do 24 hour support for this particular pile of festering - umm - "stuff".
So, at 3am one fine morning, my beeper went off. Bleary-eyed after a night on the beers, I hop into a taxi, and off to work I go. The order processing batch job has fallen over. So, find out what's causing it to crash, go in, manually remove the offending order line, schedule the job to run again, go and chat with the operators. Half an hour later, *blam*, down she goes again (these were big orders, and being processed on a small minicomputer). Super. Trudge upstairs from the machine room, find the offending line, edit it out, reschedule, back downstairs again and nip out back for a smoke. Three quarters of an hour later, *blam*. Find offending line, edit out, note that the item code is the same for all 3 lines I've now removed, make a preemptive strike on all lines with that item code, reschedule. I'm now royally bored with the whole thing, so start working out what the item actually *is* (can't do this on the live system, of course, so have to restore a backup to a test environment, involves much trudging up and down stairs looking or tapes that actually work). Turns out it's cocktail sticks. The ordering system has changed, and instead of putting in orders by the box (100,000 per box or so), it's putting in orders by the unit. So previously reasonable orders of (say) 100 boxes of cocktail sticks are coming through with a volume field of 10,000,000, which number is too big for the internal integer representation used by the program.
No, of course we didn't have anything resembling up to date source for the program. Took me 2 weeks to reverse engineer it from the binary.
 skew binary, for example