Opened 7 years ago

Closed 6 years ago

#6850 closed defect (fixed)

Record does not update its display

Reported by: bemasc Owned by: erikg
Priority: blocker Milestone: 8.2.0 (was Update.2)
Component: camera-activity Version:
Keywords: record blocks:8.2.0 Cc: olpc-dev@…, mtd, ffm, erikg, mstone, dsd
Blocked By: Blocking: #7383
Deployments affected: Action Needed: package
Verified: yes

Description

In joyride-1825, I start Record. It shows me the image from the camera, but this image, once the program has started, never changes. It does not update, unless I switch between tabs (e.g. photo to video), in which case it grabs a new frame, and then doesn't update it.

The logs show no obvious errors. The mic and camera lights remain on for the whole session.

Attachments (3)

Video.log (5.9 KB) - added by mikus 6 years ago.
Joyride 1894 - RecordActivity log
RecordActivity-1.log.log (412 bytes) - added by riverrun 6 years ago.
Record activity log: joyride-1891
testgst.py (329 bytes) - added by erikg 6 years ago.

Download all attachments as: .zip

Change History (33)

comment:1 Changed 7 years ago by erikb

  • Cc olpc-dev@… added

comment:2 Changed 7 years ago by bemasc

Hey, this is still completely broken in Record-54 in latest joyride. Record doesn't work at all. I'd love to try and contribute a fix, but I can't even tell whether 54 is near the latest development in your repo.

Please fix this. Some deployments are now choosing to base their builds off of Joyride, so it's important that Record work. It's impossible to demo Record in the "new sugar".

Changed 6 years ago by mikus

Joyride 1894 - RecordActivity log

Changed 6 years ago by riverrun

Record activity log: joyride-1891

comment:3 Changed 6 years ago by mtd

  • Cc mtd added

comment:4 Changed 6 years ago by garycmartin

  • Milestone changed from Never Assigned to Update.2
  • Priority changed from normal to high

Tested in joyride 1946 (Record v54). Still broken and unusable. You get sporadic image updates switching between tabs, but usually just a thin horizontal strip of an image. If you try recording a movie or taking a photo it'll just sit there with the wait cursor cycling until you quit the activity, no images taken.

comment:5 Changed 6 years ago by ffm

  • Cc ffm added
  • Milestone changed from Update.2 to Update1.1
  • Priority changed from high to blocker

comment:6 follow-up: Changed 6 years ago by erikg

comment:7 Changed 6 years ago by erikg

  • Cc erikg added

comment:8 in reply to: ↑ 6 Changed 6 years ago by erikg

Replying to erikg:

See also http://dev.laptop.org/ticket/2833.

I'm getting the same error message: "cafe1000-ccic 0000:00:0c.2: Frame overrun on 1, frames lost" in the kernel logs.

comment:9 Changed 6 years ago by erikg

It appears that the problem is not in Record (its source has not changed in ages), but somewhere deeper in the stack.

I have run a simple using gst-launch-0.10 to verify that the performance of the gstreamer subsystem is reduced on the newer build. On joyride-1998, the average time (averaged over 10 runs) to run "gst-launch-0.10 v4l2src ! ffmpegcolorspace ! pngenc ! filesink location=/tmp/foo.png" is 9.98s. On stable-703: 2.91s.

I am attaching my test script.

Changed 6 years ago by erikg

comment:10 Changed 6 years ago by erikg

On stable-703:

  • I ran "gst-launch-0.10 v4l2src ! ximagesink": Colorspace borked as expected. Framerate fine.
  • I ran "gst-launch-0.10 v4l2src ! ffmpegcolorspace ! ximagesink": The colorspace is correct. The framerate is lower, but not unacceptable. Processor usage is consistently around 90%.

On joyride-1998:

  • I ran "gst-launch-0.10 v4l2src ! ximagesink": Colorspace borked as expected. Framerate fine.
  • I ran "gst-launch-0.10 v4l2src ! ffmpegcolorspace ! ximagesink": The colorspace is correct. The framerate is abysmal. I'd estimate ~1fps. Processor usage sits at 10-15%.

Conclusions:

The addition of the ffmpegcolorspace filter into the pipeline caused a massive slowdown in framerate. The reduced processor usage under joyride-1998 suggests that one of the recent updates to the ffmpegcolorspace gst plug-in (there have been several in the past three months) introduced a bug/feature which causes gst to under-utilize the processor on the XO, thus producing the lower-than acceptable performance we're seeing in Record. Something is sleeping or waiting in ffmpegcolorspace where it shouldn't be.

comment:11 Changed 6 years ago by mstone

  • Cc mstone added

comment:12 follow-up: Changed 6 years ago by bert

Interesting observation - but doesn't Record use xvimagesink, making ffmpegcolorspace unnecessary?

comment:13 in reply to: ↑ 12 Changed 6 years ago by erikb

Replying to bert:

Interesting observation - but doesn't Record use xvimagesink, making ffmpegcolorspace unnecessary?

If you want to take a picture or encode a video, I am pretty sure you need ffmpegcolorspace but would be pleased to learn otherwise.

comment:14 follow-up: Changed 6 years ago by cjb

It's correct (as far as I know) that ffmpegcolorspace isn't required for Xv, which is what we use in Record.

Erik, try "gst-launch-0.10 v4l2src ! xvimagesink"?

comment:15 in reply to: ↑ 14 ; follow-up: Changed 6 years ago by erikg

Replying to cjb:

It's correct (as far as I know) that ffmpegcolorspace isn't required for Xv, which is what we use in Record.

Erik, try "gst-launch-0.10 v4l2src ! xvimagesink"?

See above. It works fine (nice framerate) but gives us psychedelic output.

comment:16 Changed 6 years ago by bemasc

Under joyride-1896, this command produces totally normal, non-psychedelic colors, because both sides are speaking YUV. Given that Record does not work in joyride-1896, this does not seem to be the issue at hand.

comment:17 follow-up: Changed 6 years ago by bemasc

also, I think erikg may not have noticed the distinction between ximagesink and xvimagesink, which are entirely different.

comment:18 in reply to: ↑ 17 Changed 6 years ago by erikg

Replying to bemasc:

also, I think erikg may not have noticed the distinction between ximagesink and xvimagesink, which are entirely different.

Exactly! You and cjb are correct.

It works fine.

comment:19 in reply to: ↑ 15 Changed 6 years ago by erikg

Replying to erikg:

Replying to cjb:

It's correct (as far as I know) that ffmpegcolorspace isn't required for Xv, which is what we use in Record.

Erik, try "gst-launch-0.10 v4l2src ! xvimagesink"?

See above. It works fine (nice framerate) but gives us psychedelic output.

Blatantly wrong on my part. I didn't see the (subtle) yet important distinction between ximagesink and xvimagesink.

comment:20 follow-up: Changed 6 years ago by bert

One data point: "gst-launch v4l2src ! xvimagesink" starts up significantly faster in update.1-708 than in faster-2019 (in faster, it takes about 3 seconds whereas in the stable build it's under 1 sec). Also, the rendering seems to interfere with the compositing (the terminal window contents appears in front of the xv display, but the video is visible where the sugar tool bar should be).

comment:21 in reply to: ↑ 20 Changed 6 years ago by erikg

Replying to bert:

One data point: "gst-launch v4l2src ! xvimagesink" starts up significantly faster in update.1-708 than in faster-2019 (in faster, it takes about 3 seconds whereas in the stable build it's under 1 sec). Also, the rendering seems to interfere with the compositing (the terminal window contents appears in front of the xv display, but the video is visible where the sugar tool bar should be).

See above. I already attached a test script which can be used to verify this.

comment:22 Changed 6 years ago by erikg

I have solved the issue by downgrading two components. I understand that this is not an ideal fix, but at least we have isolated the components (gstreamer and gstreamer-plugins-base) which give rise to this bug.

To downgrade a joyride system:

1) Get the correct rpms using koji:

~% koji download-build gstreamer-plugins-base-0.10.12-4.2.olpc2 --arch="i386"
~% koji download-build gstreamer-0.10.12-1.olpc2 --arch="i386"

2) Drop the rpms into an XO running a joyride build. We are downgrading from 0.10.15 -> 0.10.12. On the XO:

~% rpm -Uvh --oldpackage gstreamer-plugins-base-0.10.12-4.2.olpc2.i386.rpm
~% rpm -Uvh --oldpackage gstreamer-0.10.12-1.olpc2.i386.rpm
~% ldconfig

You have to run ldconfig to reconfigure the dynamic linker cache. Without doing so Sugar will not boot (on startup it tries to load pygst and thus one of the gstreamer shared libraries).

Now Record updates its display as expected.

Moving forward, I would like to test newer versions of all the gstreamer packages to see if the clocking issue which appears to be giving rise to this problem is fixed in later builds. I understand that between 0.10.12 and 0.10.15 there were changes in the hierarchy of clock manipulation in gstreamer.

Speculation: If these changes were not properly incorporated into the elements (tees queues, filters) referred to in the pipelines in Record, we might expect clocking problems as we see here.

comment:23 Changed 6 years ago by erikg

  • Keywords record added
  • Owner changed from erikb to erikg
  • Verified set

comment:24 Changed 6 years ago by dsd

  • Cc dsd added

comment:25 Changed 6 years ago by dsd

  • Action Needed set to never set
  • Blocking 7383 added
  • Milestone changed from 8.1.1 (was Update1.1) to 8.2.0 (was Update.2)

I spoke with some gstreamer developers, this is likely because of a bad pipeline. You cannot link and unlink elements in a pipeline without considering pad blocking. Also you cannot have certain elements in a pipeline in unconnected state (even if you block their pads), which we do (for example) with the filesink when we are showing live video in photo capture mode.

At their suggestion I've been working the pipeline into components using the element API (rather than parse_launch which is a fairly cryptic format and quite confusing for large pipelines). I then add + link the components into the pipeline when we need them. This is a nicer code structure and allows us to use the same core pipeline all of the time, making switching between capture modes (photo/video/audio) much smoother as well.

I'm working at http://dev.laptop.org/git?p=users/dsd/record;a=summary - nothing really there yet, but I should have some of the above changes committed later today.

comment:26 Changed 6 years ago by dsd

Posted my work in progress to git://dev.laptop.org/users/dsd/record

Video recording doesn't work quite right. Also the picture-in-picture is broken in video playback mode. It's not possible to use Xv for both playback and capture simultaenously, which is presumably why the non-Xv ximagesink was used for the PIP in that mode, but I'm having trouble getting ximagesink and xvimagesink to play nice when used in the same application.

comment:27 Changed 6 years ago by mstone

  • Keywords blocks:8.2.0 added

comment:28 Changed 6 years ago by dsd

This is fixed in latest git (users/dsd/record).

Also video recording is working again (except that you can only record one per session, #7452).

I also removed the PIP in video playback mode, replacing it with a "Back" button, but I have some ideas for how to restore this later.

comment:29 Changed 6 years ago by dsd

  • Action Needed changed from never set to package

comment:30 Changed 6 years ago by dsd

  • Resolution set to fixed
  • Status changed from new to closed

Record-55 released.

Note: See TracTickets for help on using tickets.