rahlquist
Posts: 149
Joined: Tue Jan 21, 2014 1:02 pm

Annotate, any hope for future changes?

Wed Jan 28, 2015 2:32 pm

6by9 did a stellar job IMHO getting Annotate in before he had to leave Broadcom. That said, is there likely any chance over the course of this year we will see any further changes in the abilities of this feature? Of particular interest to me is the ability to set the foreground color of the font and the ability to select a larger font, be it flexible sizing or just a few larger presets. Is there anyone in a position to be able to comment?

Thanks!

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7140
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Annotate, any hope for future changes?

Wed Jan 28, 2015 3:26 pm

I'm still here, and courtesy of the very nice Dom and gsh I still have access to the firmware code :D
What I don't have is lot of spare time, or the excuse to do it at the office, so don't expect immediate responses.

Text colour can be adjusted fairly easily at the low level, although it is specified in terms of YUV and not RGB. I can see a little script being required to do the necessary conversions for people.

Having had a very quick look at the code, there is a define TEXTHEIGHT currently set at 32. The function description for the routine to draw the text says
Size is the maximum height and must be between 6 and 160 inclusive.
The font is antialiased and at some sizes will look blurry.
So it sounds like that could be exposed relatively easily to give some configuration options.
I'll add both things to my list, and they'll happen when they happen.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

rahlquist
Posts: 149
Joined: Tue Jan 21, 2014 1:02 pm

Re: Annotate, any hope for future changes?

Wed Jan 28, 2015 4:43 pm

6by9 wrote:I'm still here, and courtesy of the very nice Dom and gsh I still have access to the firmware code :D
What I don't have is lot of spare time, or the excuse to do it at the office, so don't expect immediate responses.
Well thats totally understandable!
Text colour can be adjusted fairly easily at the low level, although it is specified in terms of YUV and not RGB. I can see a little script being required to do the necessary conversions for people.

Having had a very quick look at the code, there is a define TEXTHEIGHT currently set at 32. The function description for the routine to draw the text says
Size is the maximum height and must be between 6 and 160 inclusive.
The font is antialiased and at some sizes will look blurry.
So it sounds like that could be exposed relatively easily to give some configuration options.
I'll add both things to my list, and they'll happen when they happen.
That would be awesome. And I promise not to ask when you will have time! ;)

My biggest issue right now is just the visibility of this in the preview. http://imgur.com/1nOMNlx gives an example.

Also is it likely the below issue is in the raspimpeg that silvanmelchior created or is it possibly a firmware issue. The shift in background happens when recording starts and goes away when it ends.

http://imgur.com/HOChYsS

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23366
Joined: Sat Jul 30, 2011 7:41 pm

Re: Annotate, any hope for future changes?

Wed Jan 28, 2015 4:48 pm

It's not added new features, but I pushed a new version of the raspicam code to my github yesterday that implement annotate in to the demo apps. Allows custom text, date, time and the camera parameters to be set from the command line. Raspvid has extra stuff to display H264 parameters (and time updates on the video)

Also adds the intrarefresh type to the command line for raspivid. I was going to PR it, but it's mixed up with a previous commit that I need to sort out.

https://github.com/JamesH65/userland/tree/new_features



As for new annotate features, the big one for me would be allowing more than 32 characters in the custom text section. 32 really isn't enough to get much information when trying to annotate H264 parameters. 6x9, happy to help out if you need some minion work.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7140
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Annotate, any hope for future changes?

Wed Jan 28, 2015 5:13 pm

rahlquist wrote:My biggest issue right now is just the visibility of this in the preview. http://imgur.com/1nOMNlx gives an example.
The tricky bit is that the height is specified in pixels, so a value that works well on preview may look tiny on stills capture. I may be able to add a different size value for the two pipes, but I'm not 100% sure on the consequences of doing so.
rahlquist wrote:Also is it likely the below issue is in the raspimpeg that silvanmelchior created or is it possibly a firmware issue. The shift in background happens when recording starts and goes away when it ends.

http://imgur.com/HOChYsS
I wasn't aware of that one. Almost certainly a firmware issue.
When doing video encode we actually change the image format to one that the codec prefers - it is laid out bizarrely in memory to optimise SDRAM access on motion searches and the like. I suspect that the algorithm to do the text writing is then just writing to that image format incorrectly. Another one for the list!
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7140
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Annotate, any hope for future changes?

Wed Jan 28, 2015 5:19 pm

jamesh wrote:As for new annotate features, the big one for me would be allowing more than 32 characters in the custom text section. 32 really isn't enough to get much information when trying to annotate H264 parameters.
You'd not noticed MMAL_PARAMETER_CAMERA_ANNOTATE_V2_T then?

Code: Select all

#define MMAL_CAMERA_ANNOTATE_MAX_TEXT_LEN_V2 256
I didn't want to break backward compatibility so had to keep the old structure. IL and RIL have been updated to only accept the bigger version, and the MMAL/RIL code will convert a V1 struct to the expanded IL struct correctly.

640kB256 chars should be enough for anyone!
jamesh wrote:6x9, happy to help out if you need some minion work.
I feel the need to say bottom at this point :D
Banana?
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23366
Joined: Sat Jul 30, 2011 7:41 pm

Re: Annotate, any hope for future changes?

Wed Jan 28, 2015 5:24 pm

6by9 wrote:
jamesh wrote:As for new annotate features, the big one for me would be allowing more than 32 characters in the custom text section. 32 really isn't enough to get much information when trying to annotate H264 parameters.
You'd not noticed MMAL_PARAMETER_CAMERA_ANNOTATE_V2_T then?

Code: Select all

#define MMAL_CAMERA_ANNOTATE_MAX_TEXT_LEN_V2 256
I didn't want to break backward compatibility so had to keep the old structure. IL and RIL have been updated to only accept the bigger version, and the MMAL/RIL code will convert a V1 struct to the expanded IL struct correctly.

640kB256 chars should be enough for anyone!
jamesh wrote:6x9, happy to help out if you need some minion work.
I feel the need to say bottom at this point :D
Banana?
Nope, will search immediately!

EDIT: weird. Not sure how I missed that - I wonder if I was using an old tree at some point and it wasn't there. Anyway, found it now. Ta.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7140
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Annotate, any hope for future changes?

Wed Jan 28, 2015 5:32 pm

jamesh wrote:Nope, will search immediately!

EDIT: weird. Not sure how I missed that - I wonder if I was using an old tree at some point and it wasn't there. Anyway, found it now. Ta.
https://github.com/raspberrypi/userland ... ff05c0c4cb
30 Aug 2014 is a familiar date for some reason. Snuck it in at the last minute I think.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23366
Joined: Sat Jul 30, 2011 7:41 pm

Re: Annotate, any hope for future changes?

Wed Jan 28, 2015 8:21 pm

6by9 wrote:
jamesh wrote:Nope, will search immediately!

EDIT: weird. Not sure how I missed that - I wonder if I was using an old tree at some point and it wasn't there. Anyway, found it now. Ta.
https://github.com/raspberrypi/userland ... ff05c0c4cb
30 Aug 2014 is a familiar date for some reason. Snuck it in at the last minute I think.
Found it, tried it, worked. Frame counter will be useful.

One thing did come up, the black background was offset from the text if I updated the annotate text during a run. Didn't have time for more than a few seconds of testing, will update tomorrow.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7140
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Annotate, any hope for future changes?

Wed Jan 28, 2015 8:33 pm

jamesh wrote:One thing did come up, the black background was offset from the text if I updated the annotate text during a run. Didn't have time for more than a few seconds of testing, will update tomorrow.
You mean like rahlquist linked to above as http://imgur.com/HOChYsS ?
Implication from his comment was that it was when encoding (ie YUVUV) and that was where I was going to look. The background is just a function of the text helper function, so I doubt just updating it would cause the issue.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
Douglas6
Posts: 4739
Joined: Sat Mar 16, 2013 5:34 am
Location: Chicago, IL

Re: Annotate, any hope for future changes?

Thu Jan 29, 2015 12:46 am

I have no idea what's being said, but I do so enjoy watching these guys work.

"I love hard work. I could sit back and watch it for hours...."

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23366
Joined: Sat Jul 30, 2011 7:41 pm

Re: Annotate, any hope for future changes?

Thu Jan 29, 2015 8:59 am

6by9 wrote:
jamesh wrote:One thing did come up, the black background was offset from the text if I updated the annotate text during a run. Didn't have time for more than a few seconds of testing, will update tomorrow.
You mean like rahlquist linked to above as http://imgur.com/HOChYsS ?
Implication from his comment was that it was when encoding (ie YUVUV) and that was where I was going to look. The background is just a function of the text helper function, so I doubt just updating it would cause the issue.
Yes, that's the one. I'll try again today to see what caused the problem.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7140
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Annotate, any hope for future changes?

Thu Jan 29, 2015 9:19 am

jamesh wrote:Yes, that's the one. I'll try again today to see what caused the problem.
I think I can see the issue on that one straight off.
Arithmetic error in computing the start point within the column for the fill, so it would start too far over and possibly splat a random other bit of the image (likely to be just further down the column from where it should be). Will test a fix as soon as I can - jamesh's updated raspivid is likely to help there :)
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23366
Joined: Sat Jul 30, 2011 7:41 pm

Re: Annotate, any hope for future changes?

Thu Jan 29, 2015 10:04 am

6by9 wrote:
jamesh wrote:Yes, that's the one. I'll try again today to see what caused the problem.
I think I can see the issue on that one straight off.
Arithmetic error in computing the start point within the column for the fill, so it would start too far over and possibly splat a random other bit of the image (likely to be just further down the column from where it should be). Will test a fix as soon as I can - jamesh's updated raspivid is likely to help there :)
Not pushed the mod for long strings yet, will do in the next hour or so.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7140
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Annotate, any hope for future changes?

Fri Jan 30, 2015 7:51 am

Incorrect background sorted. It helped that I recalled fixing it on the other development branch, just a shame it hadn't made it to the Pi one :( It doesn't help when there are two bugs lurking in there, one for all planes, and one just for UV.
I'll throw a patch Dom's way and it'll get merged when he gets a chance.

Whilst I'm looking at this corner of the code, I will try to get colours and font size working too.

edit Some bad news. The text function only writes the text to the Y plane, not the U&V ones. I'll have a quick look to see if a tweak is possible, but with chroma subsampling I can see it being a little tricky to get it clean.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7140
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Annotate, any hope for future changes?

Thu Feb 05, 2015 12:54 pm

Sorry for the delay. I've just emailed Dom a patch to fix the text background filling issue when video recording. Not sure when he'll sort out a firmware update.

The other changes I have pending with (hopefully) all the low level changes made, but haven't had time to sort out testing it via raspistill (which of course means they may blow up when I try it!).

JamesH - have you done, or are you planning to do, a PR for your raspicam changes for annotate? If so I can add on top of those rather than having to reinvent the wheel.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

rahlquist
Posts: 149
Joined: Tue Jan 21, 2014 1:02 pm

Re: Annotate, any hope for future changes?

Fri Feb 06, 2015 5:56 pm

6by9 wrote:Sorry for the delay. I've just emailed Dom a patch to fix the text background filling issue when video recording. Not sure when he'll sort out a firmware update.
Has this made it into the current build, because the background is now behaving different. When I manually start a recording it only shifts to the right about 1 pixel on the left side, but the right side extends out by quite a bit.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23366
Joined: Sat Jul 30, 2011 7:41 pm

Re: Annotate, any hope for future changes?

Fri Feb 06, 2015 8:22 pm

6by9 wrote:Sorry for the delay. I've just emailed Dom a patch to fix the text background filling issue when video recording. Not sure when he'll sort out a firmware update.

The other changes I have pending with (hopefully) all the low level changes made, but haven't had time to sort out testing it via raspistill (which of course means they may blow up when I try it!).

JamesH - have you done, or are you planning to do, a PR for your raspicam changes for annotate? If so I can add on top of those rather than having to reinvent the wheel.
I did a PR for the annotate stuff today to raspi github. If you have tried it out and it seems OK, then I can merge it. I'd prefer a second glance at it to make sure I haven't done anything stupid.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7140
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Annotate, any hope for future changes?

Fri Feb 06, 2015 8:57 pm

rahlquist wrote:Has this made it into the current build, because the background is now behaving different. When I manually start a recording it only shifts to the right about 1 pixel on the left side, but the right side extends out by quite a bit.
No, it doesn't appear to have been released as yet. Keep an eye on https://github.com/Hexxeh/rpi-firmware/commits/master or https://twitter.com/RPF_Dev_Updates
The shift totally depended on the length of the text string and the capture resolution. During video encode the image is split into 128 pixel wide columns, and the bug was bad behaviour when filling in the first and last columns.
jamesh wrote:I did a PR for the annotate stuff today to raspi github. If you have tried it out and it seems OK, then I can merge it. I'd prefer a second glance at it to make sure I haven't done anything stupid.
I'd used your ANNOTATE_V2 stuff to do my quick testing and it seemed reasonable. I'll give it the once over in a mo.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

rahlquist
Posts: 149
Joined: Tue Jan 21, 2014 1:02 pm

Re: Annotate, any hope for future changes?

Sat Feb 07, 2015 7:15 am

6by9 wrote:No, it doesn't appear to have been released as yet. Keep an eye on https://github.com/Hexxeh/rpi-firmware/commits/master or https://twitter.com/RPF_Dev_Updates
The shift totally depended on the length of the text string and the capture resolution. During video encode the image is split into 128 pixel wide columns, and the bug was bad behaviour when filling in the first and last columns.
Ahh ok that makes sense to me then because I had not edited the string yet on my new pi2 when I noted the different shift and the default string is longer than I usually use.

Thanks guys for the hard work!

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7140
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Annotate, any hope for future changes?

Tue Feb 10, 2015 8:08 am

Background fix should have been in https://github.com/Hexxeh/rpi-firmware/ ... 5266295a84 made on Saturday.
Other changes still to come.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

rahlquist
Posts: 149
Joined: Tue Jan 21, 2014 1:02 pm

Re: Annotate, any hope for future changes?

Thu Feb 12, 2015 9:22 pm

6by9 wrote:Background fix should have been in https://github.com/Hexxeh/rpi-firmware/ ... 5266295a84 made on Saturday.
Other changes still to come.
Thank you. The annotate background no longer shifts after I did a full round of updates. I do however have the whole preview shifting Up and to the Left

Before Recording
https://www.dropbox.com/s/32yz46lut2vbw ... 6.png?dl=0
After starting recording
https://www.dropbox.com/s/ggtuea3ilkotc ... 1.png?dl=0

towolf
Posts: 421
Joined: Fri Jan 18, 2013 2:11 pm

Re: Annotate, any hope for future changes?

Thu Feb 12, 2015 11:16 pm

Thanks a lot for --annotate in raspivid.

With it I can finally get a grip on the real latency. Having an RTP network stream with a frame counter just filming the screen "hall of mirrors" style, I count 6 to 7 frames of latency at 60fps, which is 100 to 116ms.

Image

About the intra refresh mode with --irefresh, how do I tell if it is working? When I start the stream I get an immediate image. I was expecting more of a screen wipe.

towolf
Posts: 421
Joined: Fri Jan 18, 2013 2:11 pm

Re: Annotate, any hope for future changes?

Thu Feb 12, 2015 11:20 pm

If I remove --irefresh I get 5-6 frames of latency on the network stream. I thought the effect would be opposite?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7140
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Annotate, any hope for future changes?

Fri Feb 13, 2015 8:52 am

rahlquist wrote:Thank you. The annotate background no longer shifts after I did a full round of updates. I do however have the whole preview shifting Up and to the Left

Before Recording
https://www.dropbox.com/s/32yz46lut2vbw ... 6.png?dl=0
After starting recording
https://www.dropbox.com/s/ggtuea3ilkotc ... 1.png?dl=0
Whole image shifting is odd. What resolutions are you running at? I'll have a go at recreating as there should be no shift at all.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Return to “Camera board”