Dave Haynie
Forum Replies Created
-
Hi… old computer engineer here (Google or Wikipedia “Dave Haynie” if you want a bio).
If you have an older CPU, you have a memory controller. Anything not covered in dust from AMD, or anything with “i” in the name of the CPU from Intel, and there’s no separate memory controller — memory is managed directly from the CPU.
For DRAM, there are read cycles and write cycles. No such thing as a verify cycle, NADA. Some PCs have parity or ECC (error check/correct) memory.. there’s about one extra cycle in there, usually. DRAM (eg, DDR, DDR2, DDR3 in most systems people still use… DDR == Double Data Rate, which means two data cycle for every clock cycle) is rarely any bottleneck.
Yeah, it’s technically much slower than your CPU. But it’s also several orders of magnitude faster than your hard drive. And particularly for video rendering, it’s not that much of an issue. Your CPU has several tiers of on-chip cache, which help out here. And they’re not tiny.. on my system, each of six cores has a 128K of L1 cache… that runs pretty much as fast as the CPU. That’s backed by 512K of L2 cache per core, which in turn is backed by a shared L3 cache (L = level). So basically, if the problem you’re working on fits in 9MB or so, the CPU never touches main memory.
Caches do other things too, though. Modern DRAM is called synchronous DRAM… a fancy way of saying it’s clocked. When you address a random memory location, that process can take 5-10 cycles, maybe more. But once you have that memory item you want, the next couple are practically free by comparison. So the CPU just grabs those, and stick ’em in the cache. Most CPUs these days have two 64-bit wide buses and run four or eight cycles for each access… so they grab memory in chunks of 256-512 bytes at a time, two a time.
The idea of a cache is local storage, like that big pile of papers on your desk that your S.O. probably thinks belong in the file cabinet. If the thing the CPU is after is already in the cache (for instance, the algorithm for encoding AVC), it doesn’t need main memory.
Then there’s writing back to main memory… modern CPUs do what they call “posted” writes. The CPU writes, and usually it appears to the CPU to take absolutely no time. That write goes into cache, and at some point later, finds its way to main memory.
When you do video encoding, there’s a pretty good chance you’ll be waiting on the CPU, and possibility you’ll be waiting on the hard drive. Main memory has very little effect — it’s part of the process, but just not a practical bottleneck in video work.
A RAM disk on a modern PC is usually a very bad idea. First of all, the modern OS is very efficient at buffering data in large blocks. This keeps HDDs reading and writing at full speed. One block to/from a hard drive is slow… a couple of megabytes or more at a time, and you’re hitting the 50MB/s-100MB/s the drive can handle. Add to that the fact that most of time, the video you’re reading and writing is fairly small… 25Mb/s for most camcorder video, 50-100Mb/s for higher-end video. Your hard drive is 15-30x faster, your memory is something on the order of 10,000x faster.
If a program runs low on memory, it’ll have to simulate memory using virtual memory. The OS tells the program it has the memory, and when it uses it, there’s a hardware trick via the memory mangement unit (MMU).. the CPU traps, a routine kicks some other resource out of memory temporarily, and then loads what you need into that freed memory. It works, but as you use more VM, your performance starts to favor that of the HDD rather than your DRAM. Bad move. A RAM disk was a decent idea when I worked on the Amiga computer back in the 1980s… when the alternative was floppies. Today, it’s probably going to slow you down, unless you have so much crazy DRAM you can map whole video files.
As for multi-core CPUs.. I have a 6-core processor, and I get 97-100% CPU utilization on rendering. I’d say that’s pretty good scaling. If you’re not seeing that, you have an I/O or memory problem somewhere… but it’s not memory speed. Could be a shortage, or maybe your HDDs are just too slow (and yeah, even a RAID can be too slow if you’re pulling too many files from it at one… but that’s a different performance problem).
As for hyperthreading…it’s a bit more sophisticated than just swapping on cache misses. But yeah, it’s not always a win. For one, you’re spreading the same caches across twice the work. That’s not as bad as it sounds if all threads are rendering, and the rendering algorithm fits in cache. But if it doesn’t, you’re far more likely to thrash the cache… maybe even just an L1 or L2 cache, but thrash you may. And of course, if there is a hard drive bottleneck, increasing the aggregate CPU performance is one way to expose that.
-Dave
-
I’m not in the least defending Windows here. Just the benefit to the consumer of commodity hardware rather than semi-proprietary hardware. Apple’s selling you the same hardware for 2x-3x the price.
The key to understanding this is realizing that Apple functions as a software company, and their hardware is just a fancy and overpriced dongle. People think Apple’s a hardware company, but Alf suggested, look at their profits. The only other companies making those kinds of margins in tech are software companies. Apple’s one of the few “hardware” companies actually good at software development… because they’re really a software company.
As for the hardware: nice mechanical design, but that’s about it. Apple’s motherboards are made in the same factories as Dell’s or HP’s, with the same chips, by the same people. Apple’s products are expensive compared to the very same spec hardware made by other quality vendors — I’m not even suggesting that Apple’s expensive because I cheap out.
For example: my HP laptop cost $1280, directly from HP. The comparable MacBook Pro (same CPU, same memory, 10GB less storage, one more Firewire port, one less USB port, same radios, same Intel system chips, comparable displays… don’t know if Apple had IPS in their laptops back in 2007, half-rather-than-full sided ExpressCard port, same camera, no card reader) cost $2999, direct from Apple. Ok, this was a few years ago… you could actually swap the batteries in both, back then, today, not an option with the Mac. And the HP is still going strong.. over time, I added 2GB RAM and swapped out the HDDs for 640GB models (two of them).
What did Apple offer here? MacOS — that’s the only reason they can charge so much for the very same thing. They’re selling the software.
Same with the iOS devices.. they’re overpriced. It’s not too obvious with tablets yet, since ALL tablets have been historically overpriced. But also largely a niche item. As these catch on with the public, this will change. Most of the other top tier companies with ARM tablets are pricing relative to the iPad. But in truth, these are not more expensive than Netbooks (cheaper CPU/SOC, less RAM, less storage, smaller battery, same display, no keyboard, add $30 cost for the touch screen, $25 if you need a cellular modem).
-Dave
-
Re-encoding is always going to deliver a slightly lower quality result. What bitrate are you shooting on the Sony? The maximum bitrate on the CX12 is 16Mb/s. Technically speaking, AVCHD on DVD has to be 18Mb/s or lower, so the motion picture browser is probably just copying your camcorder files to DVD. Basically, if you take the AVCHD structure from your flash card and drop it on a DVD in UDF 2.5 format, you have yourself an AVCHD-DVD. It’s not legal to go over 18Mb/s on DVD, but you won’t have that issue.
Sony’s default “AVCHD 1920×1080 60i” template is also rendering at 16Mb/s. And no, there’s no “smart rendering”, as Sony calls it, for AVC yet – which would apply if you shot at top quality.
You should double-check, at least, that you’re actually using that template. Check the “custom” settings, and you’ll find the bitrate used for that particular output preset. I know for some of the format, Sony has some oddly low bitrates, like 8Mb/s for full HD.
Another thing.. in the project preferences, there are several settings that can have an effect on output quality. Specifically, set render quality to “best” and pixel values to 32-bit (video levels). Your output may all of a sudden look better.
-Dave
-
Well, there are a couple of questions to ask.
First one: is it “mostly” web browsing and online games, or ALL web browsing and online games. Does she use any actual Windows applications at all?
Second one: does she just surf the net, or does she do much writing. The lack of a keyboard on the iPad isn’t bad if you’re a content consumer only, but it’s an issue if you do any real writing.
Next… the games. I don’t know about every game on Facebook (or elsewhere), but Adobe Flash is probably the most popular game environment on the web at the moment. Apple doesn’t support this on any iOS device. Maybe they have special iPad apps for some of those things, or maybe she’s going to miss out.
And finally… like all Apple products, the iPad is pretty seriously overpriced for what it is. At $499 basic, it seems “reasonably priced” to Apple fans, but that’s only because the cheapest Apple laptop starts at $999. The average non-netbook laptop sold in the USA in 2009 ran about $550, and while I haven’t seen the 2010 stats, you can bet they’ve fallen a bit. I have seen fully functional (Windows 7, low-end CPU… but still 4x-8x as fast as the one in the iPad, 2GB DRAM, etc) Windows laptops on Christmas specials as low as $300. Windows 7 laptops aren’t as “cool” to some people as the iPad, but they’re dramatically more useful. And a basic one costs less. Some examples:
https://www.tigerdirect.com/applications/SearchTools/item-details.asp?EdpNo=6491156&Sku=A180-15641
https://www.newegg.com/Product/Product.aspx?Item=N82E16834200077&cm_mmc=BFUS-_-32-_-N82E16834200077&nm_mc=BFUS
https://www.newegg.com/Product/Product.aspx?Item=N82E16834146839
https://www.newegg.com/Product/Product.aspx?Item=N82E16834157535Dell has 35% tomorrow on select models. HP has a basic laptop starting at $349 tomorrow… check out https://www.shopping.hp.com/go/CyberMonday?aoid=35252
A netbook type laptop is also an option. The main difference… no optical drive, scaled-down x86 processor, smaller (10″) screen. These are typically as low as $200, so maybe even cheaper if you shop for “Cyber Monday” sales tomorrow. Keep in mind, the typical Atom CPU is about half the performance of a Core 2 or Celeron processor per clock cycle, but it’s actually comparable to the ARM Cortex A8 per clock… and usually clocked 1.5x-2.0x faster. Sure, Windows is a bit more burden than iOS, but you also get more capable web browsers on Windows… there’s something for that CPU power to do. The main complaint of netbooks for some people is the 10″-or-so screen, vs. 15″ for discount laptops. But the iPad has the same issue.
And don’t forget, if portability isn’t an issue, a desktop is still a better deal.
Now, sure, there are some people who want the iPad for what the iPad is. If you’re just a content consumer, if there’s a value to nearly all-day battery life (vs. 2-4hrs. on a laptop or 4-8hrs on a Netbook), if you’re interested in iPhone/iPad style gaming (iOS is actually the leading platform for mobile gaming, having surpassed PSP, GameBoy, etc), etc. you may find this a reasonable thing to buy.
I was thinking about getting my wife a new low-end laptop or netbook for Christmas this year. But she’s pretty much decided she’s taking over my fairly tricked out laptop (2.4GHz dual Core2, 4GB memory, 1.2TB HDD, 17″ screen… ok, it was bit more impressive three years ago). We also have a Dell I bought at a yard sale for $8.00… perfectly functional, once I killed all the viruses and spyware. But it needs another 512MB to be useful… about $70. I think the netbook would be a better investment.
-Dave
-
Here’s how to think of it… when you edit in Vegas, Vegas turns your video into uncompressed. You never see this, since it only happens in RAM.
Some kinds of input video work with “smart-rendering”… if you render out to the same format your have coming in, and don’t make any changes to it, you’ll get the same video out without re-compressing. Otherwise, you need to re-compress.
So you drop an AVCHD video on the Vegas timeline. Vegas does processing to it… could be nothing, could be compositing with other video, but in short, Vegas sets up all the instructions for you edit. When you render, those instructions “run”. And part of that is what do with the series of uncompressed video frames created by Vegas.
When you select AVC output, you’re telling Vegas to compress to AVC. The thing that’s compressed are those in-memory frames… how they got in memory has nothing to do with the output format (except in the case of smart-rendering, which is a special-case bypass of the decompress-composite-recompress process).
Now, it can be pretty well shown that your re-rendered video at 20Mb/s is not going to look any better than your input video at 17Mb/s. In fact, it’s virtually guaranteed — some loss occurs with each re-rendering. That’s also not the right question to ask. Since you’re editing and re-rendering, the real question is whether your video looks better output at 20Mb/s than some other bitrate, and whether MainConcept AVC is better/worse for the same bitrate than Sony AVC.
-Dave
-
We are a little spoiled, being on both Vegas and Windows. Vegas supports pretty much every format, and while Windows has its faults, it’s kind of the UN of media formats.
MacOS is like main street in Western Ohio. There only cultural influence is the local one… Apple. If you’re not in an Apple format, forget about it. Even to suggest adding a CODEC (for example, you can produce super high quality output, in .mov format, in Avid DNxHD or Cineform from Vegas, but many Mac houses will demand ProRes, only because that’s Apple’s version of pretty much the same thing) makes them nuts.
It’s very unlikely that standard formats like AVC rendered in an .mp4 wrapper will have any problem on a Mac. This is kind of a given… the MPEG-4 Part 14 spec, which defined the .mp4 file structure, is based on Quicktime (.mov). They are not identical, but it’s trivial for any .mov reader to handle .mp4. Chances are, if the Quicktime player on your PC is happy with your .mp4 file, so will the Mac reader/player.
-Dave
-
Rendering time is the sum of the different times involved. If you’re on a fast computer, rendering to something like Cineform or MPEG-2 is probably at or near realtime (eg, 4min render takes 4min), probably less if you’re output format is SD.
But what else is in there. Right now, I’m rendering a special effect on a 5min segment of video…. a BCC7 temporal blur effect. Each frame that’s rendered is based on effects applied across ten HD frames… that’s never going to happen fast. In fact, for that five minutes, its run over an hour and 45min, and expects to go another hour and a half. You can take about 5min out of that for rendering (I’m rendering to 50Mb/s MXF MPEG-2… I’d normally use Cineform, but it’s broken in Vegas 10… yeah, even with the latest version of NeoScene). The rest is due to the effect plug-in. And this is on a 6-core AMD at 3.2GHz… not the slowest PC on the planet.
As for size… what’s your bitrate? That’s what determines size, and pretty much, quality as well. If you want smaller files, change the details on your rendering. WMV9/VC-1 can be nearly as small as H.264 at the same quality. If I wanted high quality WMV9 for upload to YouTube in 1080p HD, I’d probably choose something around 15-20Mb/s, half of that for 720p, as a minimum (knowing full well YouTube will transcode it). A 4min video at 20Mb/s will take up about 600MB. If you rendered for easy internet play, say 1.5Mb/s (and lower resolution, or it’ll look BAD), that’s 45MB for the 4min video.
-Dave
-
Right.
It’s also a point that, for increasingly quality, you pay increasingly more. A $4000 camera isn’t 10x better than a $400 camera at most things… but it’s better enough, when you need it to better. For professional work — consumers won’t care enough to make that an important difference.
It’s also the march of technology. The 8Mpixel sensor in that Sanyo probably is better at just about everything than the 3Mpixel sensor in the HVR-A1. Software does pixel bucketing, so you don’t get the color fringing you used to get with a single sensor. The Sony has better controls, a better lens, XLRs, etc… so it’s a better overall camera. But technology has marched on, and also, quite a bit of what you pay for on a high-end camera isn’t the imaging, it’s the other capabilities, the controls, etc.
My main camera these days is a Panasonic HMC40… low-end pro or high-end consumer, whatever you want to call it. For less than half the price, I also bought a TM700… which has an upgraded version of the same imaging path (lens, sensor) as the HMC40. Most the time, when I shoot with two cameras, one’s mounted on a tripod anyway — so any money spent for better controls is wasted. Even better audio is questionable… I have a mic on the HMC40 that was more than half the price of the TM700, and I usually have one or more stand-alone field recorders going too. So audio on the “B” camera is largely just for synch. A more expensive camera won’t give you a better image… except in low light. Of course, I could get a DSLR for about the price of a TM700 that’s going to give me better low-light results than a high-end 2/3″ camcorder… but has its own set of issues.
The nice thing is, it’s not necessarily an either/or choice. The right tool for the job, at least within budget, is why my camera/camcorder collection keeps growing.
-Dave
-
I recommended the FH1A… I have the FH1. The one difference… the FH1A has this stupid Apple “iFrame” ever-so-slightly-better-than-SD mode. Apparently they make too many Macs out of cheap laptop motherboards, they can’t handle full HD… or something. Anyway, that’s not a real problem, other than the likelihood that many consumers will buy these and think they’re shooting in HD, but won’t be.
This is a decent low-light camera for the money. It’s got an f2.0 lens, 1/2.5″ CMOS sensor. It also supports a couple of 1080/30p, 720/30p, and 480/30p modes, so you can shoot at 1/30th sec, and gain another f-stop. Also offers full manual controls… a little weird, they’re all photo style (eg, ISO settings rather than gain).
This is no match for my Pansonics, and the lens isn’t as good as my old Sony HVR-A1, but I’d give it the edge in low-light over the Sony. It’s not much larger than a “flip” style camera, and cheap enough to drag along places I wouldn’t risk a better camcorder.
Some of the better Flip-style, like the Kodak Zi8, might produce similar quality video, but they lack a zoom lens. The Zi8 is one of the few of this class with a larger sensor (also 1/2.5″), but then again, it’s an f2.8 lens, so you lose an f-stop versus the Sanyo. The Kodak, like most of that type, also has a fixed-focus lens, so no close focusing… the Sanyo has a macro mode. The Kodak does use reasonable “real-camcorder” recording bitrates, up to about 20Mb/s depending on mode… many of the Flip-style models use about half that, or less.
Just pointing it out.. it’s a little risky to recommend any consumery model like the Sanyo on a board like Creative COW, since most folks here know their stuff, and some will not be happy with any tiny pocket-sized camcorder. I figure having a camcorder with me is better than not having one, but I’d like that to be as little of a compromise as possible. Check out the reviews on this… I think there’s a full review of the FH1 or FH1A on camcorderinfo.com. While not in precisely the same class as the Zi8 or the Flips, this is under $300… Fry’s had them for $200 (now sold out), so it’s pretty much the high end of that price class. And the still photos aren’t too shabby, either.
-Dave
-
Vegas 10 plays fine with modern Panasonic cameras. I think there was a time, when the AVCHD specs were new, that interoperability was a big question. Not so much anymore.
The one problem I’ve seen: Vegas doesn’t detect TM700 1080/24p video as 1080/24p. Now, of course, Vegas wouldn’t need to “detect” anything if Panasonic wasn’t still insisting on encoding 24p as 60i with 3:2 pulldown… that should have ended with tape. And in fact, it did… older Panasonic models, such as my daughter’s SD9, did “native” 24p. So do higher-end Panasonic models, such as my HMC40.
But Panasonic seems to have decided to inflict this problem on consumers. You could make the argument back in the SD days: tape needs a constant frame rate, DVDs want 24p with pulldown anyway. But why re-introduce this bad idea to the consumer line? It is, after all, just a software thing.. they could easily change this to native 24p with a software upgrade.
On the other hand, Vegas ought to detect 24p in a 60i wrapper, or at least let you force the issue. I use Cineform for 24p I shoot on the TM700.. Cineform was a pretty good idea with AVC anyway, until recently (faster PC, better support in Vegas), but not everyone wants another $100 in software just to access a feature that they seemingly already paid for. Ok, there’s probably a freeware solution of some kind…
-Dave