I found reliable operation in cases like this needed at least a 2s battery with couple buck converters. One for Pi and one for the motors. They are very cheap from china (as low as 5 for a dollar). Attempts to use linear regulators or one cell with/without boost converters eventually failed.
Quite a few noobs here insist one cell works fine but either eventually switched to two or just disappeared. For non-critical apps it's fun to play around but if something needs to actually work better than "some of the time" then good electrical engineering practices may be worth investigating.
This sounds promising but how do I split power from the battery to the two buck voltage converters?