Sony’s New Smartphone CMOS Image Sensor Captures Full HD Video at 1000 FPS

[Megalith]

How’d they manage that? Well, the company figured out how to integrate DRAM into their CMOS image sensor—a world’s first. The additional memory layer means very fast data readout speeds, which allows for all sorts of neat stuff that include capturing fast-moving subjects with minimal focal plane distortion, as well as super-slow-motion movies at up to 1,000 frames per second.

…the sensor's standout feature is super slow-mo. As shown below, 1,000 fps is pushing into Phantom Flex camera territory, letting you see the impact of a ball on a bat or a dog vault in precise detail. What's more, Sony says that smartphones could detect sudden movement and automatically launch the high-speed mode, so you only use it when needed. Thanks to the DRAM buffer layer, it would work on any smartphone with a regular image processor. By contrast, the Google Pixel, one of the better slow-mo models out there, can only do 120 fps at full HD, less than an eighth of the Sony sensor's capability.

 

[Mugato]

What amazes me is watching both the dog and horse jump at the precise moment to get over those hurdles. The dogs belly seems to only brush the rod and the horses foot is a mm above. It takes athletes thousands of hours to achieve that, and they still fall in the olympics. Neat to watch in slo-mo!

 

[Jim Kim]

Can i get this as an option on my tracphone?

 

[captaindiptoad]

Holy shit!

 

[Deleted member 184142]

Goodbye storage space.

 

[lilbabycat]

1:15 Was worried that dog's dog would hit the bar. Whew

 

[Retronym]

this is going to be used for a whole different kind of slow mo.

 

[NeoNemesis]

What amazes me is watching both the dog and horse jump at the precise moment to get over those hurdles. The dogs belly seems to only brush the rod and the horses foot is a mm above. It takes athletes thousands of hours to achieve that, and they still fall in the olympics. Neat to watch in slo-mo!

Agreed. It probably has to do with us overthinking and overanalyzing everything.

 

[raz-0]

Want. Would definitely be an improvement over my casio's 1200fps that has a resolution that makes everything look like soeone is shaking a colander full of pea gravel.

 

[cyclone3d]

The article is full of crap... From what they are saying, current cameras can only capture images at 1/30th of a second.

Pretty much ANY DSLR and pretty much ANY film camera for the last 20 or so years can do a 1/1000th - 1/2000th of a second.

Most newer DSLRs can do a max of 1/4000th to 1/8000th of a second shutter speed.

My current one, Pentax K-30 can do 1/6000th of a second shutter speed.

Now, 1000fps 1080p video on a cell phone is really nice... wonder how goo the lighting has to be to actually get that though, especially on a super tiny cell phone camera sensor with a super tiny cell phone camera lens.

 

[AgentxofxChaos]

Pretty cool demo. They should have done some Baywatch style demonstrations too though, just sayin'

Would definitely love to have that feature in my phone but am curious to how large the resulting files size is. Granted it's a year or two away, and I'm sure storage capacity on phones will significantly increase, and that might not be as big of an issue even if it is significant now. Regardless, I'm still curious about it.

 

[Spidey329]

The article is full of crap... From what they are saying, current cameras can only capture images at 1/30th of a second.

Pretty much ANY DSLR and pretty much ANY film camera for the last 20 or so years can do a 1/1000th - 1/2000th of a second.

Most newer DSLRs can do a max of 1/4000th to 1/8000th of a second shutter speed.

My current one, Pentax K-30 can do 1/6000th of a second shutter speed.

Now, 1000fps 1080p video on a cell phone is really nice... wonder how goo the lighting has to be to actually get that though, especially on a super tiny cell phone camera sensor with a super tiny cell phone camera lens.

I didn't see them mention 1/30 at all. Are you talking about?:

images in just 1/120th of a second, "four times faster than conventional products," Sony says.

Conventional implies other phone cameras (e.g. conventional for this form factor). Not DSLR's with large sensors and shutter systems.

Additionally, shutter speed doesn't necessarily coorelate to frames per second in video. DSLR's may be able to snap a shutter at 1/8000, but they usually can't consistently for a full second as they bump up against memory bottlenecks.

 

[cyclone3d]

I didn't see them mention 1/30 at all. Are you talking about?:

images in just 1/120th of a second, "four times faster than conventional products," Sony says.

Conventional implies other phone cameras (e.g. conventional for this form factor). Not DSLR's with large sensors and shutter systems.

Additionally, shutter speed doesn't necessarily coorelate to frames per second in video. DSLR's may be able to snap a shutter at 1/8000, but they usually can't consistently for a full second as they bump up against memory bottlenecks.

Sony has unveiled a sensor that could bring some impressive camera tricks to your next smartphone. The 3-layer CMOS sensor does super slow motion at up to 1,000 fps in full HD (1,920 x 1,080), around eight times faster than any other chip. That's possible thanks to a 2-layer sensor married with high speed DRAM that can buffer images extremely rapidly. Specifically, it can capture 19.3-megapixel images in just 1/120th of a second, "four times faster than conventional products," Sony says.

That kind of readout speed reduces "focal plane distortion," also known as rolling shutter. On CMOS-equipped cameras, including smartphones and DSLRs, the top of the an image is read before than the bottom, causing vertical lines to tilt on fast moving objects. As Sony shows in a test image (below) a faster 1/120th second readout speed significantly reduces that effect. The result will be better photos of moving objects and reduced wobbly "jello" video.\

So, either the article writer hasn't a clue about what they are writing about or they are trying to intentionally mislead people.

 

[raz-0]

So, either the article writer hasn't a clue about what they are writing about or they are trying to intentionally mislead people.

I'l suspect that it's a teensy bit of sony marketing and a LOT of the author doesn't know.

I suspect in part, that confusion is being caused by talking about the whole package. Yeah, sensors can capture at shutter speeds of 1/8000th and such, but they can't push crap across their memory bus to intermediate and/or durable storage faster than about 18fps without dumping data. Most cameras doing HD video are cropping well below that point and cutting you off at 1080p24, which can get you jello vision.

Whatever it is the net result is clearly improved performance in high frame rate video applicaitons... by a lot. Especially if the price point is smack dab in consumer territory.

 

[Deleted member 184142]

The article is full of crap... From what they are saying, current cameras can only capture images at 1/30th of a second.

Pretty much ANY DSLR and pretty much ANY film camera for the last 20 or so years can do a 1/1000th - 1/2000th of a second.

Most newer DSLRs can do a max of 1/4000th to 1/8000th of a second shutter speed.

My current one, Pentax K-30 can do 1/6000th of a second shutter speed.

Now, 1000fps 1080p video on a cell phone is really nice... wonder how goo the lighting has to be to actually get that though, especially on a super tiny cell phone camera sensor with a super tiny cell phone camera lens.

This does not have to do with shutter speed. The shutter when shooting video is not the mechanical shutter, as most will never even hit 30fps, no less 1,000fps. Taking a single still vs a streaming video are NOT the same thing. In cases of video on a DSLR it uses an electronic shutter that is a function of the sensor only, by a type of discharge. This has to do with memory and being able to handle the sort of bandwidth and file size this would put out. Mechanical shutters are used for stills because even with a 1/1000th electric shutter, it is still sensitive to light because while it can capture those 1/1000th shutter speeds, without a mechanical shutter to close, you get all sorts of funny distortions like jello effect (rolling shutter), because even though the shutter has those high speeds, it still takes time to clear the sensor, as with a CMOS each row is cleared at a different time. So yes, this IS faster than most sensors on even DSLR's when not accounting for the mechanical shutter assistance in blocking light for those high shutter speeds. Shutter speed is also about double the effective frame rate, so for 1,000FPS you would need a 1/2000 shutter speed to not change the "feel" of the video.

This is another reason some cameras have a cap on burst photos and slower shot speed, even though the sensor can do a 1/4000, it does not have the memory or processing speed to handle those rates. Some global shutters with VERY high read outs and memory to support it can get away without a mechanical shutter and can in some ways replicate the effect of a mechanical focal plane shutter, these are expensive however and have some limitations. This new sensor seems to have the same ability with very fast readout speeds, but in a very small package for cheap, this is not easy, because this can over heat sensors very easy when working at these rates for video and the bandwidth required.

 

[cyclone3d]

This does not have to do with shutter speed. The shutter when shooting video is not the mechanical shutter, as most will never even hit 30fps, no less 1,000fps. Taking a single still vs a streaming video are NOT the same thing. In cases of video on a DSLR it uses an electronic shutter that is a function of the sensor only, by a type of discharge. This has to do with memory and being able to handle the sort of bandwidth and file size this would put out. Mechanical shutters are used for stills because even with a 1/1000th electric shutter, it is still sensitive to light because while it can capture those 1/1000th shutter speeds, without a mechanical shutter to close, you get all sorts of funny distortions like jello effect (rolling shutter), because even though the shutter has those high speeds, it still takes time to clear the sensor, as with a CMOS each row is cleared at a different time. So yes, this IS faster than most sensors on even DSLR's when not accounting for the mechanical shutter assistance in blocking light for those high shutter speeds. Shutter speed is also about double the effective frame rate, so for 1,000FPS you would need a 1/2000 shutter speed to not change the "feel" of the video.

This is another reason some cameras have a cap on burst photos and slower shot speed, even though the sensor can do a 1/4000, it does not have the memory or processing speed to handle those rates. Some global shutters with VERY high read outs and memory to support it can get away without a mechanical shutter and can in some ways replicate the effect of a mechanical focal plane shutter, these are expensive however and have some limitations. This new sensor seems to have the same ability with very fast readout speeds, but in a very small package for cheap, this is not easy, because this can over heat sensors very easy when working at these rates for video and the bandwidth required.

They are also talking about still images though when they are comparing 1/30 and 1/120. "The result will be better photos of moving objects and reduced wobbly "jello" video"

And flash sync speed is even higher than that at 1/180. Some cameras can do higher.

The readout speed may have something to do with taking video, but the way the article is written, it makes it sounds like the current tech is absolute trash even for still images which just isn't true.

It is physically impossible to get that image angle skew in pictures when you have a shutter speed high enough to STOP the action.

 

[Deleted member 184142]

They are also talking about still images though when they are comparing 1/30 and 1/120. "The result will be better photos of moving objects and reduced wobbly "jello" video"

And flash sync speed is even higher than that at 1/180. Some cameras can do higher.

The readout speed may have something to do with taking video, but the way the article is written, it makes it sounds like the current tech is absolute trash even for still images which just isn't true.

It is physically impossible to get that image angle skew in pictures when you have a shutter speed high enough to STOP the action.

You are still comparing cameras with a mechanical assistance which stops all light. This sensor, if true would be at the top of the top in sensors readout times. shutter speed does not matter if you do not have a mechanical shutter to assist (such as video or still cameras without a mechanical shutter), as you might have a 1/4000 shutter speed but it takes another 1/100 to clear the sensor and readout, in that time the sensor is still taking in light, which can cause other distortions, so without a mechanical shutter, your actual shutter speed is dependent on readout speed. Remember, this sensor is doing these speeds without being a global shutter (CCD) and without a mechanical shutter and as a CMOS sensor, that is impressive. You are getting closer to the actual shutter speed of a CCD, but without the cost, most DSLRs with a good sized CCD sensor come in around $5,000, with most full frames around 10-15k. While it still misses a few check boxes a CCD has, it's one step closer. As with faster real shutter speeds will also result in better images and less noise.

Trust me, if DSLRs could pull out those kind of FPS for video, we would have that as a setting or it would be available in one of the FW mods out there, many of them don't even do 120FPS and are well into the $5,000 range.

 

[Xrave]

Give me a 1000fps money shot.

 

[daglesj]

Stocks in super high speed camera manufacturers plummeted today...

 

[hardboner]

phones of the future will be fuuun,
and as usual, Apple will be the last to incorporate this

 

[Poseur]

I'm interested. Cell phones, conventional digital cameras, video cameras and security cameras could all benefit. The video suggests it needs lots of light though. No surprise, but a bummer too.