Stereo camera PI processing

[worldmaster]

Hello. Please tell me how to do the following:
I have two cameras that are connected to the Raspberry PI.
I need to get an original size image from each camera. For example, 1024 * 1024.
Then cut the fragment size 640 * 480.
Mirror these two fragments and merge them into one image of 1280 * 480 size.
Then stream the stream through rtsp to the network.
Compression required h264. FPS 30.

I read that raspivid can set the region of interest and work with 2 cameras simultaneously in stereo.
But can he process the frames individually to solve my problem?
Please help with the solution. Thanks.

[mahjongg]

Compute modules have two camera inputs, and software exists to combine the two camera's for stereo vision, and split screen display of both camera's

Using regular RPI's with two camera's is much harder, and involves a camera multiplexer, and the above software doesn't work.

[HermannSW]

There is a way to connect two cameras to a single PI, can be any two Arducam cameras, or Raspberry v1/v2 cameras:
https://www.raspberrypi.org/forums/view ... 8#p1551503

Then cut the fragment size 640 * 480.
Mirror these two fragments and merge them into one image of 1280 * 480 size.

The 2MP monochrome global shutter camera has a 640x400 mode.
That would give you 1280x400 size image without any work.
Above image is 2560x800 (two 1280x800 2MP images side-by-side).

P.S:
Scaled down image allows to see 3D image without any viewer (on 15" laptop display).
Just place hand vertically in the middle and make both images overlay:

[worldmaster]

I do not need the camera mode 640 * 480. I need to cut a fragment from each camera. A fragment can be anywhere in sight.

[gordon77]

"I have two cameras that are connected to the Raspberry PI."

What are these 2 cameras ?

[worldmaster]

"I have two cameras that are connected to the Raspberry PI."

What are these 2 cameras ?

https://onpad.ru/catalog/cubie/raspberr ... /1864.html
Производитель: WaveShare
Поддерживает: Raspberry PI A+/B+/PI2
Сенсор: 5 мегапиксельный сенсор OV5647
Размер матрицы: 1/4
Диафрагма (F): 2,8
Фокусное расстояние: 3.37 mm
Диагональ: 72,4 градусов
Разрешение: 1080p
- 2592 × 1944 Разрешение изображения
Размеры: 25 х 24 мм х 9 мм
Поддерживает запись видео: 1080p30, 720p60 and 640x480p60/90

[gordon77]

How are they connected?

[worldmaster]

How are they connected?

Stereopi
https://www.crowdsupply.com/virt2real/stereopi

[HermannSW]

I do not need the camera mode 640 * 480. I need to cut a fragment from each camera. A fragment can be anywhere in sight.

Then using stereo hat with v1 camera is what can do that.
The stereo hat hides the fact that two cameras are connected.
You would use raspivid and it will think it takes a 5MP image.
In that image the left half will be from left camera, and right half from right.
Then just use raspivid's "--roi" option to specify region of interest.
Of course that does not make sense if you already use StereoPi.

[gordon77]

The stereopi site mentions using raspivid,

raspivid -3d sbs -w 1280 -h 480 -o 1.h264

can you use - roi with that?

[worldmaster]

I repeat, I need fragments from each of the cameras. These fragments must be combined into a new frame. And then send the data stream to the network.

[gordon77]

Roi will select a segment BUT I don't know if it will work for 3d, and select a different segment from each camera (which l don't think you mentioned before)

[HermannSW]

I repeat, I need fragments from each of the cameras. These fragments must be combined into a new frame. And then send the data stream to the network.

I have to repeat, the Stereo hat hides the fact that there are two cameras, Raspberry Pi thinks there is only one camera.

Roi will select a segment BUT I don't know if it will work for 3d, and select a different segment from each camera (which l don't think you mentioned before)

Again, Stereo hat provides "normal" frames, Pi does not know about two cameras connected, nor is any 3D option required for raspistill/raspivid.

But I was wrong with "-roi", since the Pi does not know about two cameras, it does not work the way needed.

This is simple stereo capture for this command:

Code: Select all

$ raspistill -md 4 -w 1296 -h 972 -p 22,50,648,486 -o p1.jpg
$

Now with "-roi" I get different regions:

Code: Select all

$ raspistill -md 4 -w 1296 -h 972 -p 22,50,648,486 -o p2.jpg -roi 0.25,0.0,0.5,0.5
$

Besides having received free samples of 0.3/1.0/2.0MP monochrome global shutter, 13MP/16MP color rolling shutter and 0.3MP color/2.0MP monochome stereo cameras from Lee at Arducam, I have no incentive in increasing Arducam sales.

Btw, Arducam stereo hat for Raspberry Pi with two 5MP camera bundle does cost 89.99€
https://www.uctronics.com/index.php/ard ... -2173.html

Comparable StereoPi starter kit does cost 125$ but includes a compute module 3+ lite:
https://www.crowdsupply.com/virt2real/stereopi

So if you already have a Pi, stereo hat is cheaper.

Stereo hat with two 8MP v2 cameras does cost 109.99$.
In addition to StereoPi it allows to get stereo hat with two Arducam cameras as well (all those I listed above).
I do have a sample here at home with two ov9281 monochrome global shutter cameras (my previous posting photo was taken with that, a "2 x 1280x800 = 2560x800" frame):
https://stamm-wilbrandt.de/en/forum/ArduCAM/mode7.jpg
The Arducam cameras do provide full and not half resolution as v1/v2 cameras.
But they only work with Arducam driver, not with raspivid/raspistill like the v1/v2 cameras:
https://github.com/ArduCAM/MIPI_Camera


So "-roi" does what it should above, capture 50% size in both dimensions , starting at coordinate (324,0).
What is needed for the OP request is to be able to specify two regions of interest.
I will come up with such a solution based on raspividyuv and i420toh264 tool (with GPU doing .h264 conversion) soon:
https://github.com/Hermann-SW2/userland ... i420toh264
What I can say for sure now is that that solution will work, and will allow to capture two arbitrarily located regions of interest.

[HermannSW]

So "-roi" does what it should above, capture 50% size in both dimensions , starting at coordinate (324,0).
What is needed for the OP request is to be able to specify two regions of interest.
I will come up with such a solution based on raspividyuv and i420toh264 tool (with GPU doing .h264 conversion) soon:
https://github.com/Hermann-SW2/userland ... i420toh264
What I can say for sure now is that that solution will work, and will allow to capture two arbitrarily located regions of interest.

Done and commited, with new section added to README.md with details:
https://github.com/Hermann-SW2/userland ... g-pipeline

This command part of bash i420toh264 pipeline (x1,y1,x2,y1,w,d)

Code: Select all

...
./sample_yuv_dbl_roi 640 480 40,160,371,51,240,180 |
...

extracts regions with left top (40,160) and (371,51) from 640x480@90fps raspividyuv video, both of dimension "w x d", and creates new frames of dimension "2w x d" from them for further processing. There are quite some assertions on coordinates, width, height, ... -- the 2nd frame has to be right of left frame (x1+w<=x2) for easy inplace copying to work, besides that their top left coordinates can be selected freely.

This is a frame from generated 480x180 video tst.h264:



For comparison, this is full 640x480 view:

[worldmaster]

Thanks. And how now to broadcast the received stream to the network?

[HermannSW]

That is a completely different question.
Depends how/where you want to receive it.
In browser, vlc, raw .h64 streaam, using rtsp:, ....

There are tons of solutions, many here in forum, or on the web.
You can send and receive with gstreamer pipeline via several protocols.
Or this:
http://frozen.ca/streaming-raw-h-264-fr ... pberry-pi/

In case you want to do it with i420toh264 bash pipeline, you just need to know that that tool accepts "-" as filename in order to send the GPU compressed .h264 stream to stdout like raspivid[yuv], and form there you can use any method for broadcast.

[worldmaster]

Thanks for answers. Maybe you can tell a ready-made solution so that the stream received from i420toh264 is broadcast via rtsp with h264 compression?

[HermannSW]

Thanks for answers. Maybe you can tell a ready-made solution so that the stream received from i420toh264 is broadcast via rtsp with h264 compression?

Replace "raspivid ... -o -" with "raspividyuv ... | ... | ./i420toh264 - width height" for options 1/2/4
https://raspberrypi.stackexchange.com/q ... 1429#31429


From
https://raspberrypi.stackexchange.com/q ... tcat#27083

you can change option 2 above with:

Code: Select all

... | cvlc -vvv stream:///dev/stdin --sout '#standard{access=http,mux=ts,dst=:8090}' :demux=h264

and access on client browser with:

Code: Select all

http://<IP-OF-THE-RPI>:8090

[worldmaster]

Thank you so much. You have helped me a lot. I will try.