Hey camera vendors - less firmware, more software

Thom Hogan pointed out something I hadn't thought about, and it got me thinking - why isn't the software in our cameras getting smarter faster?

UPDATE - coincidentally, I came across this article that Samsung is experimenting with Android as a camera OS. Way to go, Samsung. Nikon, Canon, are you listening? I'm not saying it has to post to Facebook after taking a picture. I'd just like a more flexible and common architecture that would make it easier to add features - either by you or third parties.

So I was procrastinating from some work I need to do (as opposed to writing this?), getting jonesin' for some good camera scuttlebutt since the D4/D800/1DX/5DMkIII hasn't shipped yet, and was getting down to the individual photoblogger pages. Thom Hogan brought up a nice point in the midst of a piece along the lines of Get Over Your Gear, Just Go Shoot Something Good. He said this:

That said, as usual both Nikon and Canon have managed to include enough irritations in their new products that there's definitely something to complain about from everyone. I'm still convinced that the camera designers don't really know how people are shooting (or perhaps trying to dictate how people shoot). I'll cite one example: HDR. On the Nikons, you have exactly one choice: create a two-shot HDR JPEG or not. If you choose not, the bracketing system will fight you in setting up exactly the sequence you want. Neither option (automated or on your own) gives you any help in defining where the "edges" of your scene's brightness are. In other words, none of the engineers seem to actually have shot an optimized HDR sequence. They do know how to combine a dark and light image, though, thus the JPEG two-shot combo. I'll reiterate my offer: I'm willing to take a dozen Japanese engineers out shooting for a few days and show them the way things are actually done in the field. My only goal would be to make sure that the camera engineers actually here real user demands.


And my first thought was YES. Some things we figure out and get into and just stop thinking about why it is so convoluted, we're just happy we can do it.

And it sometimes takes someone else walking up and asking "Why is that so hard and complicated to do?"

Good question.

When I shoot HDR timelapse of sunsets, for instance, my methodology is to find the brightest detail I want to hold detail in (the sun and surrounding sky). To do this, I start with Aperture Priority, dial in the stop I want, and take a shot. I note the exposure duration, flip over to full manual (which I'll be using for shooting anyway), then manually recreate those settings. Then I take a series of test shots, closing down the stop until I find I'm holding detail in as much of the sky/sun as I care to. Then, since I'm a man and can't multitask worth a damn, I count stops on my fingers while I'm doubling the exposure four times in a row to figure out what four stops brighter would be. And no, I can't divide thousandths of a second by sixteen in my head, smartypants. Anyway, I use that number as my baseline exposure for a 9 shot bracketed set. Yep, this means the top end is blowing out to pure white, but by my methodology, That Is OK for my timelapsing plans.

In any case...that is a full paragraph on my technique. Pay attention, Nikon and Canon - WHY THE HELL IS THAT SO MUCH WORK?

My second thought was an HDR Assist Mode - the camera takes a quick series of low res shots at different exposures, you pick the darkest and brightest you want to use from a 3x3 or 4x4 grid, then tell it how many shots you want in the series (shot priority) or how many EV between each shot (spacing priority) and then the camera figures it out from there.

My third thought - Then I realized I'm a control freak geek and it doesn't need to be anywhere near that complicated. Duh. The camera has all kinds of fancy metering equipment - that is its job. So why isn't it doing THIS job? There should be a means of telling the camera "I want to shoot HDR. I want to not clip highlights in more than X% of the frame, and I don't want to clip shadows in Y% of the frame, Go Figure It Out For Me." If it can be done in one frame, it does it. If it will take bracketing and HDR, it warns you first to be sure you're stable, then does it. And saves the results as more than an 8 bit JPEG, for God's Sake!!!

Or just automate the thing entirely and it spits out an image as .EXR, .HDR, 8 or 16 bit TIFF, or JPEG, if that is all you want.

My fourth thought - Why are truly NEW features in cameras so slow to evolve?

These days, ALL cameras are built out of the following:
-a sensor, stuck in the middle of a computer
-a lens bolted on the front
-buttons sprinkled liberally over the body, proportional to the amount you pay

That's it.

Sensors - clearly there's all kinds of aggressive work being done in this space. Well done.

Lenses - they're on it. Keep calm and carry on and whatnot.

Buttons (and form factors) - I love that the industrial design is still a priority, and little nubs and ridges and textures and angles of buttons are continually refined. Lots of energy going into it.

The computers themselves - clearly getting more powerful - Nikon can push 3x the pixels through a D800 as compared to a D700, but at the same frame rate and with a deeper buffer. Canon bumps their fps as well, and adds not only lateral but axial chromatic aberration correction (and yeah, that counts as a software feature, but in the context of image quality, one of the obvious priorities for these guys).

The software - and here's where I take issue - features are refined, but not that many are ADDED. And the software controls? Pages of settings with drill down menus were new...in the early 90s.

And where's the action in hardware/software these days? The action isn't in desktops, it is in handheld. Tiny computers with advanced flexible user interfaces and, most importantly, flexible, readily programmable software environments that make it easy to add new features - for the OEM or third parties.

If you're going to ask for up to $1000 more for this camera than the last one, how about, gee, I dunno - get CREATIVE with the new features?

Features are creeping forward very slowly in DSLRs. 3 1/2 years later, while the 5D Mark III is improved in many ways, there is little that is NEW. Headphone jack? Arguably should've had it the first time. HDR mode? Ah, there's a brand new feature! Intervalometer? Nope. Still not. Nikon has been building them in for years, Canon wants to sell you a $135 piece of hardware (the TC-80N3). This is old school thinking at its worst. Rather than get creative and build in a solution that integrates better with the functionality of the camera, sell them a piece of dedicated hardware. Are there some advantages to the dedicated hardware approach? Yes. Are there MORE advantages to writing a few lines of code into the firmware? I'd think so. Why haven't they done this? In a bigger sense, why are we buying $3000-$6000 devices that still have a crappy interface from the early 1990s? Painfully linear drill down menus? Pages of settings? Seriously? I understand it is an embedded system. I understand the wisdom of leveraging a common hardware/software UI across a product line, and not every model can financially support a costlier solution.

But it is Time To Change. I realize I'm playing fast and loose (and demanding a lot) with industry definitions of firmware vs software, but look at it from a user standpoint. If you same "firmware" - that sets a certain level of expectation about what can be accomplished. Namely, a limited expectation. If you say the word "software" to someone, especially in the context of a portable device, that sets an expectation for a much more capable device and feature set - an expandable feature set.

Creating firmware, to the user side of my brain, is about implementing controls for the hardware at hand. Firmware is about "Yes, we have this awesome hardware, I just need a way to set the damn setting or make sure when I push this button/twiddle this knob the proper changes are made to how the hardware will be controlled." I emphasize that word because that is so much of how firmware came to be - everything used to be hardwired - you push the shutter release, a mechanical or electrical impulse makes the shutter go. Eventually, things got contextual - what happens when I spin this dial? Depends on what mode I'm in. How does the device know what mode it is in or what spinning the dial does? Drat, time to go electronic, or we might have to show a UI on a screen now. Firmware speaks to me of the PITA of making the device go ahead and do what the hardware was made to do.

Software, on the other hand, makes me think about what CAN I make the hardware do that it might not have been originally designed to do. What features can I add that based on clever manipulation of the hardware at hand?

With the exception of allowing a timing change to add 24p to the 30p only Canon 5D Mark II, when was the last time you heard of someone ADDING a feature to a camera that already shipped? Firmware updates fix problems, but pretty much never add new features. When Nikon added some profiles to their cameras, I'd argue that was just loading new presets, not adding new functionality. Software updates? New versions of software? Those add new features all the time! If you're fixing a bug? Yes, please, ship it to me for free, it should've worked that way when you shipped it. Adding new features? Now we can have a conversation about what it might cost. I like Red's approach of shipping all software (word chosen intentionally) for free - why not increase the value of your customers existing cameras? Won't that make them happy customers, likely to come back in the future?

Off the top of my head, features implementable in software without changing the hardware could be:
-automated HDR as described above
-intervalometer improvements - Nikon - hello, add a digit to that 999 limit? And Canon? HAVE ONE AT ALL.
-Nikon - if you're going to have HDR mode, and you're going to have timelapse straight to video mode, why can't I combine those modes? Obvious.
-Canon - match that feature!
-and so on! And beyond what I, with my own vertical needs, might add

So vendors, stop coding firmware, and start thinking about writing software. Firmware makes me think of lowest common denominator, just enough to get done what we already know how to do, a quick means to control the hardware. Software makes me think of writing anything the hardware is physically capable of, which might be far more than the hardware enabling firmware aspired to.

So write me some automated HDR software in camera, not a firmware HDR bare bones solution.*

That image at the top of this article? Someone sent me that image - it is cell phone design before and after the iPhone. No one told the other manufacturers they'd have to do it this way, they just started doing it.

Think about how fast, creative, and usefulness expanding all the camera apps for Android and iPhone have become - and this is because there was an easy way to both code and distribute modifications/additions to the existing hardware/software Apple & Android partners shipped. Just yesterday, Apple shipped an update that lets me access the camera in a single gesture from the phone, instead of 2 or 3. No new phone, just a few minutes of my attention to update their software (yeah yeah yeah technically firmware but you better get my point).

Hell, Canon - you've ALREADY got your own users clamoring to do this with the Magic Lantern project. So does Panasonic! I hear there's a Nikon 5100 hack making progress too. Why see this as a threat when your own customers are trying to improve the cameras they've already bought? Help them! And if you're worried about your market segmentation getting damaged because you've been differentiating (aka limiting) via firmware not hardware...I gotta say I'm not too sympathetic - time to start rethinking along modern lines of what CAN be done, instead of what you can ENFORCE to be the status quo.

And start thinking some more about writing features in software, rather than executing hardware controls via firmware. HDR is a good example of you folks getting there. Look at the iPhone and Andriod and what they can do compared to cell phones of 6 years ago. They made a programmable platform with open ended controls and open ended programming capabilities.

Apple added a considerable amount of smarts to a previously "dumb" category of product (phones), and eventually, when they were ready, opened the platform to other developers. What might a Nikon or Canon platform, or eventually app store, look like that let you download added functionality for you cameras? Apple isn't in a mood to share their codebase, but hey, open source Android, how you doin'?

Look where it got them.

* Canon gets significant points for doing in camera alignment and offering 4 different tone mapping modes and the option of saving the 3 source frames. That is starting to smell like software to me. Nikon, with your blend-two-frames-and-be-happy-you've-got-it solution, are you listening? But Canon reeks of firmware development by still not having a built in intervalometer - child's play to add as a feature via software, not hardware. If you can add 24p via firmware to the 5D Mark II, what's stopping you from adding an intervalometer? SERIOUSLY. "Because that's how we've always done it?" Not. Good. Enough. Any. More.