Color mode & Dynamic Range

[Andrei V]

I'm not convinced D-Cinelike is the best choice at this point either and I also have questions about using a non zero value for contrast. The video I posted was recorded on the 4th -- the same day I did the exposure tests and actually the flights were done before the exposure tests. So, I have more testing to do and it would be better to have a more contrasty day than the overcast day on the 4th. But, as I mentioned I have had to return my P4P due to the gimbal problem but I hope to have a replacement tomorrow -- sadly, the weather looks to be quite spotty here in SLC area for a few days beyond that.

I think to better quantify things it is better to do stationary tests as I did with the exposure tests on the 4th. Controlling things so that the only thing that changes is the camera settings should make comparisons easier. As in the exposure tests I did you want a scene with high contrast and then lock the aperture and ISO and control exposure by shutter speed alone. Start with it at the over exposed end and increase shutter speed until you are clearly crushing the blacks. Then repeat with different color profiles and values for contrast adjustment. Pick the exposure in each test that is just below clipping the highlights and adjust for a consistent look for all the tests then look for variations in shadow and highlight detail.


Brian

That's what I did Brian, if you take a look at the screenshots I posted above. Exposure was adjusted to just below clipping the highlights. Then, with the bird hovering, I filmed in both None and Dcinelike modes. The scenes were shot at noon, so they exhibit a pretty wide dynamic range, even though the sun isn't directly in the shot.

 

[yannick_c]

To be sure if we can have more DR in cinelike/Art or not, the best way for me is to shoot clipped highlights with manual settings, and compare how clipped they are with the different profiles. For exemple how much clouds are clipped with the different profiles.

 

[ianwood]

Looks like DJI cut and paste the same image processing algorithms from their A9 code. Which means the P4P has the same image limitations as the X5 and is best when using NONE and 0, 0, 0. And I bet the ISO gain is probably pretty bad too once you get past 400. Bummer. With such a great sensor, adding some better processing and an actual LOG mode would be really useful. DJI still has work to do.

 

[Andrei V]

None, Dcinelike and art are very similar when it comes to highlight clipping/retention, Yannick, if not identical (with matched exposure). There are more differences evident in the shadows.

 

[David_Cambridge]

I don't understand your point. The only way to test dynamic range would be to test with subjects with varying dynamic ranges. If you use a static dynamic range subject, lowering contrast or using some non-linear contrast reduction math (cine, art) will just scrunch the black points and white points closer together just as shown in the examples. The image differences will be due to the various non-linearities programmed in the specific DJI curves.


Sent from my iPad using PhantomPilots

 

[David_Cambridge]

It may be more helpful to think about the issue in two pieces. Dynamic range is required to avoid loosing data either at the white end or black end, or both. If there is no clipping visible in the waveform, there are no dynamic range problems. The DJI contrast control, exposure, and curves all effect clipping.

If no clipping is occurring the artistic preferences of the photographer come into play. In between the black point and white point the way the image tones get mapped is controlled by all of the controls being discussed. It is really an issue of taste on the part of the artist and the scene content to choose what looks good.

A workflow that works well is a simple 2 step.

1. Make sure no clipping is occurring in the scene you are photographing.

2. Adjust the transfer curve to please you in whatever way works best for you. My preference is to make this adjustment in post processing where the tools are much more powerful.


Sent from my iPad using PhantomPilots

 

[phneeley]

Thanks again Brian and Andrei for all the work you're sharing with the community. Much of it is admittedly way above my head

WRT DJI's useless implementation of D-log, are older generations of the Phantom (P3 models) affected, too, or is this something new with the P4P?


Sent from my iPhone using PhantomPilots

 

[Dronefriend]

It may be more helpful to think about the issue in two pieces. Dynamic range is required to avoid loosing data either at the white end or black end, or both. If there is no clipping visible in the waveform, there are no dynamic range problems. The DJI contrast control, exposure, and curves all effect clipping.

If no clipping is occurring the artistic preferences of the photographer come into play. In between the black point and white point the way the image tones get mapped is controlled by all of the controls being discussed. It is really an issue of taste on the part of the artist and the scene content to choose what looks good.

A workflow that works well is a simple 2 step.

1. Make sure no clipping is occurring in the scene you are photographing.

2. Adjust the transfer curve to please you in whatever way works best for you. My preference is to make this adjustment in post processing where the tools are much more powerful.


Sent from my iPad using PhantomPilots

I'm sorry but you are mistaken. The thing is, D-Log and the Constrast option does not affect camera input. It only affects output.

If I set the P4P to record a black chair in front of the sky, with the sun shining straight into the lens, (you get my point here), the only difference choosing d-log or contrast -3 or -1 would do is that the spectrum would clip at around 15-90 IRE. The black would look grayish, the sky would look gray, but it would still be crushed with no way to recover. This is because the camera see's the same thing regardless of mode and contrast setting. The settings only affects what part of the spectrum the codec decides to put the information at. In a real, decent camera, these settings would mean the information reaching the codec would get treated different depending on if it's shadow information or highlight or midtones. This is not the case here: same treatment, just put at a narrower range in the spectrum. Meaning even MORE crushed.

Edit: More to it.

And it does matter, even if we don't get clipping. When we're dealing with such a small sensor and fairly low bitrate we want to utilize all the storage we can. We want to fill the waveform as much as we can, because the fuller the gradient, the better it will look.

If I gave you a painting in only black and white, but with very detailed information there. I then give you two options: You may choose to save a copy with 5 different levels of black > white. Or 20 different levels. This is the same thing as using a fuller part of the waveform. The more levels we have from the beginning, the more gradient info is stored and the more natural it will look. Using -3 contrast is like choosing only 5 levels. How are you or the editing program supposed to know the remaining 15 levels when you want to "restore" it to the fuller waveform in post? You cant!

 

[David_Cambridge]

I don't agree with most of your post. Dynamic range issues are just as I described. You are correct that a compressed capture uses a more limited bit range and as a result has fewer gray steps between the max white and max black. This is an arguable problem, as the human eye is incapable, by a lot, of differentiating 256 discrete grey steps.


Sent from my iPad using PhantomPilots

 

[Tomas Wangen]

As promised, here are the result of a controlled superhigh-contrast test I did today.

Setup: f 5.6, iso 100, shutter prioritizing shadows (manually adjusted), h 264 and h 265, none, dcinelike and dlog (six combinations)

Main findings:

1) There is no difference in captured contrastrange between h264 / h265. Only captured detail is affected (which I will post results for later)

2) I am extremely impressed with the dynamic range captured. I have added one creative grade just to indicate what is possible to extract (from a none-profiled capture).

3) dlog profile, as currently implemented, is utterly useless.

4) The only observable difference between dCineLike and none is somewhat lifted shadows in dCineLine which seemingly does not keep more detail in the shadows. The color-tone is also slightly different. Highlights are the same. Choosing between dCineLike and none should therefore be a matter of taste and both are acceptable from a dynamic range perspective (according to me at least)

Edit 5) Raw file shows the same cut-off point for highlights as None, indicating that we are getting the maximum dynamic range as is.The distribution of tones can ofcourse still be improved with a working dlog profile.

Following this post, I will add the same captures with baseline correction (contrast only) to indicate how the limited range of dlog more or less destroys the endresult. Again, very small differences between dCineLike and none as others have indicated.

 

[Tomas Wangen]

And here are the contrast adjusted captures. Note that I was not able to match saturation for the dlog sample and notice how the highlights and shadows suffer in same sample.

 

[Roamer105]

We can say that independent of exactly how wide the captured dynamic range is, we do manage to reproduce absolute black and absolute white from recorded data in post production. So aside from a camera maximizing the width and detail shades of the dynamic range, it will never be totally satisfactory to represent reality. By default the dynamic range is captured linear, but ultimately restricted to 8 bits - or 256 shades. For example, if I want to focus on a lot of shadow details, I need to overemphasize the dark parts of the image by loosing the brightest parts (overexpose). The same is true the other way around. On the other hand, the different (D) modes effectively act like a lens focusing (like zooming in) on a part of the "real/linear" dynamic range to be photographed. D-modes are non-linear capturing modes that effectively distort (stretch some areas of the dynamic range and compress others) still fitting the captured shades it into 8 bits. Without post production (applying a corrective LUT and grading), the recorded video is inherently bad or at a minimum not representative of what had been photographed. However, by using a LUT, it is possible to expand the dynamic range into a wider spectrum than would have been recorded in linear mode.

One of the real issues with any form of compression is the amount and exactly what data to discard to produce the recorded media. The algorithms and codec parameters used in the drone(s) requires a critical compromise between the amount of input data, speed of recording (fps) and available processor power. From observations and comparisons, it is clear that the encoder is rather simplistic when it comes to shaving information. For example, reducing sharpness, contrast and/or saturation causes the magnitudes of significant picture information to be reduced to a point where it drops below some arbitrary threshold in the encoder causing it to be irretrievably lost. I found it is better to "stress" the encoder by not reducing input data, instead possibly increasing input data (such as landscape style (+1,+1,0) and use D-Cinelike or D-Log. It is easy to reduce sharpness in post production if so desired, the reverse, attempting to retrieve it does not really work - it will simply tend to enhance mpeg artifacts.

From what I've read here it does seem odd how the P4P modes work. Looks like DJI will be busy upgrading firmware for quite a while. As far as the older cams such as P3P are concerned, there has been a change with the latest firmware upgrade affecting the camera and modes. But clearly, the (D) modes on the P3P are working exactly as they should.

 

[yannick_c]

Bought a P4Pro yesterday, made some tests today to compare the profiles None, D-Cinelike, Art and D-Log :
f/5.6, 100 isos, shutter going from 1/50 to 1/4000 to see how the profiles behave in the highlights and the shadows.
Sharpness, contrast and saturation at 0. Shot in 4K 30fps h265 100Mbps.

D-Log is useless.
D-Cinelike : loose a little in the highlights but win a little in the shadows compared to None
Art : win a litte in the highlights and win a little in the shadows compared to None

Art is less saturated compared to None and D-Cinelike

For me Art is the winner.

All videos can be downloaded here :
part 1 : https://we.tl/5wnT6B5tMv (1.95 Go)
part 2 : https://we.tl/s1wy83dV49 (1.44 Go)

Zip file with the 12 captures : http://cerrutti.fr/temp/P4Pro/P4Pro_profils_tests.zip

 

[yannick_c]

The captures in the shadows

 

[Dronefriend]

So much testing (Y)

 

[SkyOhio]

This thread is getting intense. What conclusions have been drawn for this simpleton?


Sent from my iPhone using PhantomPilots

 

[yannick_c]

The last conclusion : after the last firmware update from today that I've just done (01.01.0203)-> no more Art profile . Ok, I thought it was the best one, now it's gone...

But new profiles: None, Truecolor, D-Cinelike, D-Log, Film-A, Film-B, ... , Film-H

I think we"ll need to make all those tests again.

 

[Dronefriend]

The last conclusion : after the last firmware update from today that I've just done (01.01.0203)-> no more Art profile . Ok, I thought it was the best one, now it's gone...

But new profiles: None, Truecolor, D-Cinelike, D-Log, Film-A, Film-B, ... , Film-H

I think we"ll need to make all those tests again.

Hahaha this is so ironic.

Here's to hoping they have improved them, a lot!

 

[Raptorman0909]

The last conclusion : after the last firmware update from today that I've just done (01.01.0203)-> no more Art profile . Ok, I thought it was the best one, now it's gone...

But new profiles: None, Truecolor, D-Cinelike, D-Log, Film-A, Film-B, ... , Film-H

I think we"ll need to make all those tests again.

WOW, they're changing this stuff up with all new profiles and eliminating others -- just WOW.

So, basically, all the work we've done in the last couple weeks will need to be redone. Has D-Log been fixed (I doubt it)? What is Truecolor, Film-A, Film-B, ... , Film-H?

Crap, testing all that will be hugely labor intensive -- one wonders if this is DJI's attempt to throw us off the case of there terrible image processing by pulling the table cloth off the table every few months.

The new firmware appears to be only for the P4P+, but with changes to the color profiles then we should expect the P4P to be getting new firmware shortly as well.


Brian

 

[yannick_c]

I got the P4P, so the new update from today (01.01.0203) is also for it.

blade_strike from rcgroups just said they will add Art mode back soon.

For the other modes, no idea what they are and if they have changed the D-Log mode. Will do some more tests tomorrow (it's 1am here so not the best time for this) after testing how the drone flies (I didn't even started the motors yet)