Record does not update its display

[bemasc]

In joyride-1825, I start Record. It shows me the image from the camera, but this image, once the program has started, never changes. It does not update, unless I switch between tabs (e.g. photo to video), in which case it grabs a new frame, and then doesn't update it.

The logs show no obvious errors. The mic and camera lights remain on for the whole session.

[bemasc]

Hey, this is still completely broken in Record-54 in latest joyride. Record doesn't work at all. I'd love to try and contribute a fix, but I can't even tell whether 54 is near the latest development in your repo.

Please fix this. Some deployments are now choosing to base their builds off of Joyride, so it's important that Record work. It's impossible to demo Record in the "new sugar".

[mikus]

Joyride 1894 - RecordActivity log

[riverrun]

Record activity log: joyride-1891

[garycmartin]

Tested in joyride 1946 (Record v54). Still broken and unusable. You get sporadic image updates switching between tabs, but usually just a thin horizontal strip of an image. If you try recording a movie or taking a photo it'll just sit there with the wait cursor cycling until you quit the activity, no images taken.

[erikg]

See also http://dev.laptop.org/ticket/2833.

[erikg]

Replying to erikg:

See also http://dev.laptop.org/ticket/2833.

I'm getting the same error message: "cafe1000-ccic 0000:00:0c.2: Frame overrun on 1, frames lost" in the kernel logs.

[erikg]

It appears that the problem is not in Record (its source has not changed in ages), but somewhere deeper in the stack.

I have run a simple using gst-launch-0.10 to verify that the performance of the gstreamer subsystem is reduced on the newer build. On joyride-1998, the average time (averaged over 10 runs) to run "gst-launch-0.10 v4l2src ! ffmpegcolorspace ! pngenc ! filesink location=/tmp/foo.png" is 9.98s. On stable-703: 2.91s.

I am attaching my test script.

[erikg]

On stable-703:

• I ran "gst-launch-0.10 v4l2src ! ximagesink": Colorspace borked as expected. Framerate fine.
• I ran "gst-launch-0.10 v4l2src ! ffmpegcolorspace ! ximagesink": The colorspace is correct. The framerate is lower, but not unacceptable. Processor usage is consistently around 90%.

On joyride-1998:

• I ran "gst-launch-0.10 v4l2src ! ximagesink": Colorspace borked as expected. Framerate fine.
• I ran "gst-launch-0.10 v4l2src ! ffmpegcolorspace ! ximagesink": The colorspace is correct. The framerate is abysmal. I'd estimate ~1fps. Processor usage sits at 10-15%.

Conclusions:

The addition of the ffmpegcolorspace filter into the pipeline caused a massive slowdown in framerate. The reduced processor usage under joyride-1998 suggests that one of the recent updates to the ffmpegcolorspace gst plug-in (there have been several in the past three months) introduced a bug/feature which causes gst to under-utilize the processor on the XO, thus producing the lower-than acceptable performance we're seeing in Record. Something is sleeping or waiting in ffmpegcolorspace where it shouldn't be.

[bert]

Interesting observation - but doesn't Record use xvimagesink, making ffmpegcolorspace unnecessary?

[erikb]

Replying to bert:

Interesting observation - but doesn't Record use xvimagesink, making ffmpegcolorspace unnecessary?

If you want to take a picture or encode a video, I am pretty sure you need ffmpegcolorspace but would be pleased to learn otherwise.

[cjb]

It's correct (as far as I know) that ffmpegcolorspace isn't required for Xv, which is what we use in Record.

Erik, try "gst-launch-0.10 v4l2src ! xvimagesink"?

[erikg]

Replying to cjb:

It's correct (as far as I know) that ffmpegcolorspace isn't required for Xv, which is what we use in Record. Erik, try "gst-launch-0.10 v4l2src ! xvimagesink"?

See above. It works fine (nice framerate) but gives us psychedelic output.

[bemasc]

Under joyride-1896, this command produces totally normal, non-psychedelic colors, because both sides are speaking YUV. Given that Record does not work in joyride-1896, this does not seem to be the issue at hand.

[bemasc]

also, I think erikg may not have noticed the distinction between ximagesink and xvimagesink, which are entirely different.

[erikg]

Replying to bemasc:

also, I think erikg may not have noticed the distinction between ximagesink and xvimagesink, which are entirely different.

Exactly! You and cjb are correct.

It works fine.

[erikg]

Replying to erikg:

Replying to cjb:

It's correct (as far as I know) that ffmpegcolorspace isn't required for Xv, which is what we use in Record. Erik, try "gst-launch-0.10 v4l2src ! xvimagesink"?

See above. It works fine (nice framerate) but gives us psychedelic output.

Blatantly wrong on my part. I didn't see the (subtle) yet important distinction between ximagesink and xvimagesink.

[bert]

One data point: "gst-launch v4l2src ! xvimagesink" starts up significantly faster in update.1-708 than in faster-2019 (in faster, it takes about 3 seconds whereas in the stable build it's under 1 sec). Also, the rendering seems to interfere with the compositing (the terminal window contents appears in front of the xv display, but the video is visible where the sugar tool bar should be).

[erikg]

Replying to bert:

One data point: "gst-launch v4l2src ! xvimagesink" starts up significantly faster in update.1-708 than in faster-2019 (in faster, it takes about 3 seconds whereas in the stable build it's under 1 sec). Also, the rendering seems to interfere with the compositing (the terminal window contents appears in front of the xv display, but the video is visible where the sugar tool bar should be).

See above. I already attached a test script which can be used to verify this.

[erikg]

I have solved the issue by downgrading two components. I understand that this is not an ideal fix, but at least we have isolated the components (gstreamer and gstreamer-plugins-base) which give rise to this bug.

To downgrade a joyride system:

1) Get the correct rpms using koji:

~% koji download-build gstreamer-plugins-base-0.10.12-4.2.olpc2 --arch="i386"
~% koji download-build gstreamer-0.10.12-1.olpc2 --arch="i386"

2) Drop the rpms into an XO running a joyride build. We are downgrading from 0.10.15 -> 0.10.12. On the XO:

~% rpm -Uvh --oldpackage gstreamer-plugins-base-0.10.12-4.2.olpc2.i386.rpm
~% rpm -Uvh --oldpackage gstreamer-0.10.12-1.olpc2.i386.rpm
~% ldconfig

You have to run ldconfig to reconfigure the dynamic linker cache. Without doing so Sugar will not boot (on startup it tries to load pygst and thus one of the gstreamer shared libraries).

Now Record updates its display as expected.

Moving forward, I would like to test newer versions of all the gstreamer packages to see if the clocking issue which appears to be giving rise to this problem is fixed in later builds. I understand that between 0.10.12 and 0.10.15 there were changes in the hierarchy of clock manipulation in gstreamer.

Speculation: If these changes were not properly incorporated into the elements (tees queues, filters) referred to in the pipelines in Record, we might expect clocking problems as we see here.

[dsd]

I spoke with some gstreamer developers, this is likely because of a bad pipeline. You cannot link and unlink elements in a pipeline without considering pad blocking. Also you cannot have certain elements in a pipeline in unconnected state (even if you block their pads), which we do (for example) with the filesink when we are showing live video in photo capture mode.

At their suggestion I've been working the pipeline into components using the element API (rather than parse_launch which is a fairly cryptic format and quite confusing for large pipelines). I then add + link the components into the pipeline when we need them. This is a nicer code structure and allows us to use the same core pipeline all of the time, making switching between capture modes (photo/video/audio) much smoother as well.

I'm working at http://dev.laptop.org/git?p=users/dsd/record;a=summary - nothing really there yet, but I should have some of the above changes committed later today.

[dsd]

Posted my work in progress to git://dev.laptop.org/users/dsd/record

Video recording doesn't work quite right. Also the picture-in-picture is broken in video playback mode. It's not possible to use Xv for both playback and capture simultaenously, which is presumably why the non-Xv ximagesink was used for the PIP in that mode, but I'm having trouble getting ximagesink and xvimagesink to play nice when used in the same application.

[dsd]

This is fixed in latest git (users/dsd/record).

Also video recording is working again (except that you can only record one per session, #7452).

I also removed the PIP in video playback mode, replacing it with a "Back" button, but I have some ideas for how to restore this later.

[dsd]

Record-55 released.