-
Video performance requirements analysis?
As a methodology that applies to videos in general, not one single video one time, how can one estimate whether or not their video for playback on computer will play smoothly on a roughly average computing platform, not over a network?
I know the “dumb” way, which would be to test each video in isolation, as if it were the only video I have ever seen, on a roughly average computing platform, never accumulating knowledge about what traits seem to have contributed to whether or not it plays back adequately.
Not that.
What are metrics that can be used to judge the efficiency of a video, other than video bandwidth?
-
16 Replies
-
[Kendall Shaw] “how can one estimate whether or not their video for playback on computer will play smoothly on a roughly average computing platform”
I don’t think there’s a single metric since many factors influence video performance, including but not limited to: video dimensions, codec, bit rate, potential hardware bottlenecks: GPU, RAM, bus and disk speed.
-
MPEG-2 and MPEG-4 attempt to address this question with levels and profiles. The idea there is that a device promises support for a specific profile and level, and any content encoded according to those specs should work.
https://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Profiles
Walter Soyka
Designer & Mad Scientist at Keen Live [link]
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
@keenlive | RenderBreak [blog] | Profile [LinkedIn] -
A problem is that the target is not a device that a profile was created for, I think. It’s a computer.
Also, I have seen varying performance using the same profile.
But, I hadn’t thought of looking at what those profiles and media encoder profiles do. That probably is a good starting point to look for factors affecting performance.
[Brian Charles] “I don’t think there’s a simple metric since many factors influence video performance, including but not limited to: video dimensions, codec, bit rate, potential hardware bottlenecks: GPU, RAM, bus and disk speed.”
Each of those is a metric. I want to come up with something more specific than each of those and figure out how you would emulate it though. For example, what is average disk speed and how much does it affect video playback performance, in isolation.
I would think that companies that produce a lot of video would have automated tests of some sort as part of QA, instead of relying entirely on people manually checking video to see if they think it looks right.
This is a made up scenario, it’s not the motivation for my question: A different way to think about the factors is if one were asked at a job interview, how would you ensure that videos shipped as video files on CD-ROMs playback consistently for the average consumer? Presumably the applicant would not answer “you can’t” or give some vague answer. Or, the applicant would want to demonstrate to the interviewer that they are aware of general methods to address common problems with video encoding.
-
[Kendall Shaw] “This is a made up scenario, it’s not the motivation for my question: A different way to think about the factors is if one were asked at a job interview, how would you ensure that videos shipped as video files on CD-ROMs playback consistently for the average consumer? Presumably the applicant would not answer “you can’t” or give some vague answer. Or, the applicant would want to demonstrate to the interviewer that they are aware of general methods to address common problems with video encoding.”
Assuming average means they have a computer from the last five years (not a fair assumption if we’re actually talking about CD-ROM drives), I’d answer that bit rate was the critical factor for CD-ROM delivery, because the access speed of the CD-ROM is likely be to the first limiting factor.
As long as you are sticking close to video standards, it’s my experience that nearly any modern computer can play 1080p H.264 locally, and nearly any modern computer with a broadband connection can play streaming/progressive download 720p H.264. I stick with AME’s Vimeo presets for client delivery with no problems.
What is the actual motivation for your question? Speaking practically and generally, I think that video encoding and playback is a solved problem in 2015.
This may be easier to discuss given a concrete scenario than it is in the general case.
Walter Soyka
Designer & Mad Scientist at Keen Live [link]
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
@keenlive | RenderBreak [blog] | Profile [LinkedIn] -
Typically the thread goes off the rails when specifics are brought up. But, the main motivation is that I observe that I automatically seeks a methodical solution to almost any task I am involved with, so it\’s there in my head nagging at me.
One of the nagging thoughts is how to make the process less time consuming when judging whether my after effects compositions are playing back correctly or not.
The most common problem that I deal with is that rendered video plays back in fits and starts. Sometimes uncompressed video plays back smoothly while using youtube 1080p profile mp4 in media encoder the video plays in fits and starts. Sometimes the opposite is true.
The same goes for video bitrate. Sometimes a video file with vastly higher video bitrate plays smoothly for me, while a lower bit rate video file plays in fits and starts. Sometimes the opposite is true.
If one has a faster computer, these become less apparent. But, that is no solution since the audience for the video is people who are not me and who don\’t have my computer.
So far, I have developed superstition about what affects video playback quality. Then when my routine fails I develop a new set of superstitions. I want that to stop and to know what I am doing instead.
I would also like to be able to capture knowledge of what factors affect playback quality into automated tests.
-
[Kendall Shaw] “So far, I have developed superstition about what affects video playback quality. Then when my routine fails I develop a new set of superstitions. I want that to stop and to know what I am doing instead. I would also like to be able to capture knowledge of what factors affect playback quality into automated tests.”
I guess you need to consider all the ways that video playback can fail. I think there are a few major categories:
Insufficient bandwidth. Data cannot be read off the disk/over the network fast enough to keep the decoder’s buffer full; frames are dropped when data is unavailable. This is often a problem with uncompressed media; although uncompressed is trivially easy to decode, it has massive bandwidth requirements and very few disk systems can actually keep up. Note that with poorly-constrained variable bit rate encoding, it’s possible for the average bit rate to be within spec for your disk or network, but allow temporary bit rate spikes that exceed available bandwidth. This will result in dropped frames.
Insufficient compute availability. Decode for some codecs is extremely CPU-intensive. If the frames cannot be decoded at the rate by which they are needed, they will be dropped. Note that compression generally trades file size against encode and/or decode complexity; the smaller a file is (while maintaining quality), the more powerful CPU necessary for decode.
Bad data. A buggy encoder or a transmission error may cause invalid or missing data which forces the decoder to produce an error.
Display. If the application cannot push the rendered frames onto the GPU fast enough, you’ll drop frames. If playback spans multiple independently-clocked monitors, you’re likely to drop frames. If the GPU driver hiccups, you may drop frames.
The thing that complicates these when talking about a general purpose computer is that these situations may arise temporarily and without warning. If you’re in a low-memory situation and the system starts swapping to disk, you will lose bandwidth unexpectedly. If you’re running many applications simultaneously, you may have occasionally suffer reduced compute capacity.
If you want to geek out on the media file side of this, check out ffprobe. There’s automated QC software from vendors like Tektronix, but it’s pricey. Network shows go through QC at dedicated facilities. I watch my output because it goes out the door with my name on it.
Walter Soyka
Designer & Mad Scientist at Keen Live [link]
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
@keenlive | RenderBreak [blog] | Profile [LinkedIn] -
I want to figure out how to translate factors like those into judging the quality of a video. Video playback can be affected by CPU speed, for example. A corresponding test would be to determine somehow how CPU speed is affecting the playback of some video and what it is about the video that is being affected by the CPU speed.
The opposite is easier, presumably, given some video see if it’s playback is improved by a faster CPU. This approach works to fix the playback of one video for one person, one time.
The point of all this is to have the video playback for other people with unknown computer configurations. When CPU speed affects playback it is a characteristic of the video that allows for CPU speed to affect the playback poorly. That characteristic is what I’m after, for each of however many factors there are.
I have used ffprobe/avprobe before, but aside from finding out what type of video I have, I haven’t known what to do with the information for diagnosing a problem. The same goes for bitrate viewer. The documentation says it helps you to find and get rid of “bad stuff”. The identifying of this “bad stuff” and knowing what to do about it is what I don’t know and what I would have the software for in the first place.
You’ve given me what seems like 2 good ideas at least though, I can use the low level information that I won’t understand coming out of ffprobe on different video files and compare them to see if I can start learning to recognize problems that way. The other is to see what the rationale for hte different mpeg profiles is and to look at the sample test case code for mpeg that’s a public standard.
-
what about jpeg2000? it supports adaptive playback with layers upon layers of data to increase or decrease the decode quality on the fly using a single file. lower quality just ignores the sub quality layers. Used from everything from
hospital imagery to Cinema 4K profiles DCP. You might not even need to re-encode!https://www.3dtv.at/Products/Jpeg2000Decoder/Index_en.aspx
-
It certainly sounds good. My only exposure to JPEG2000 was when someone was using it to serve giant still images over the web. It’s use would depend on the application of the video. So, it is not a general solution to performance problems, I think.
My question is really about video in general. Even in a scenario in which I could use JPEG2000, I would want to know how to address this problem:
Bob has a computer and will playback a video. Video version 1 and 2 are the same content. Version 1 has an unknown problem.
Video version 1: The JPEG2000 decoder adapts to “low quality” when Bob plays the video.
Video version 2: The JPEG2000 decoder adapts to “medium quality” when Bob plays the video.I want to know how to figure out what the unknown problem is that leads the decoder to adapt to a lower quality in version 1 vs. version 2.
-
you would need a video benchmarking tool which gives benchmark numbers on hundreds of items. another problem not discussed yet, for example, the same video played in quicktime would skip but in media player home classic, the 3rd party decoder works 170% more efficiently. Neither the hd, ram, proc, bandwidth, etc. or anything else would matter.
That’s the real problem and why vimeo and quicktime suck. The poorly programed decoder…
Reply to this Discussion! Login or Sign Up