Tuesday, January 25, 2011

.svg Overlays in Cheese

I finally added the capacity for Cheese to show image overlays: any .svg file the user wants to use above the image.

This one was tricky. I started studying from zero the GStreamer API, because I had no idea what was all that about sinks, pads, bins, caps, ghost pads, linking elements, pipelines and so on. What were those things?

So I started reading beginner tutorials, the GStreamer Application Development Manual, and the GStreamer Core Reference Manual.

I was having problems getting the rsvgoverlay plugin in gst-plugins-bad to work. I was getting errors. I debugged, generated huge GST_DEBUG logs, studied the rsvgoverlay plugin code, tested different gst-launch pipelines. I slowly started to understand what was going on.

There was some problem with gst-parse-bin-from-description () and the way it created the rsvgoverlay element. I won't bore you with the details, but rsvgoverlay element has two sinks, and gst_parse_bin_from_description () creates only one ghost pad in each direction (sink or src), it says so in its documentation:

Ghost pads on the bin for unlinked source or sink pads within the bin can automatically be created (but only a maximum of one ghost pad for each direction will be created; if you expect multiple unlinked source pads or multiple unlinked sink pads and want them all ghosted, you will have to create the ghost pads yourself.

Whoa! That wasn't too easy to understand for a beginner. But now it sounds clear, it's funny.

With help from several people but especially from fujii and thiagoss we finally solved it. And here are the results:

Happy birthday bunny!

The overlays don't follow you around, but that's the plan for the future, to link somehow facedetect to rsvgoverlay x and y coordinates.

A bigger picture of the hat
(Because I made that hat) tee-hee!

We are also accepting art contributions in .svg format to include in cheese as sample effects! Christmas/halloween decorations? Balloons, flowers, hats, beards? Your imagination is the limit.

*Whispers* The users can use their own images too very easily editing the .effect text file!


Jeff Wheeler said...

The crosshairs bit weirds me out a bit, to be honest.

L. said...

@Jeff: changed it to something more colorful! I was thinking "measuring device" not firearms anyway :) http://en.wikipedia.org/wiki/Reticle

diego said...

You are my new favorite person.

Mike Sheldon said...

In terms of following faces around you should find that facedetect posts bus messages containing the values "x", "y", "width" and "height" for any faces that it finds.

So if you connect the "message::element" signal up to a function that looks out for messages called "face" you should be able to get the data you need fairly easily.

taschenorakel said...

Lovely fun feature! :-)

lucasr said...

Great work Laura!

Anonymous said...

Nice overlays!

The fact that gst_parse_bin_from_description can only create bins with one sink and one source pad is not really a problem for your case. Your effect does have only one sink and one source pad after all. Problem is it not being inteligent to choose the pads to link internly or to ghost, which can probably be fixed.

stuaxo said...

This is really cool ... @ some point I might want to look at coding similar stuff, I wonder if that text could be improved ?

Rolandixor said...

I love it! When can I make a sandwich with this cheese?

lol great work Laura you deserve a beer :D

Rolandixor said...

Any chance of tabbed effects in cheese? We could have hundreds of effects in the near future (trumping all other platforms in the process :P) - but we will need a way to organize the effects into groups (and to avoid live previews frying poor little netbooks around the world >:)

Anonymous said...


Anonymous said...

i wonder is it possible to stream this stuff into a new video device in order to use it say in skype as a camera.

That should be fun

Anonymous said...

i wonder is it possible to stream this stuff into a new video device in order to use it say in skype as a camera.

That should be fun

Anonymous said...

Cool! thats a feature i was waiting for long time. Hope you continue making so awesome work.. Best reggards from a Cuban Developer