switch_mode_and_capture_request does not restore camera settings afterwords.

[marsfan]

When I set up a camera configuration in both preview and high resolution modes, start the preview mode, change the exposure time, and then use switch_mode_and_capture_request to take a high resolution photo, after taking the photo, the preview mode has had its exposure time reset to the initial value instead of the last value that I set into it.

How can I change this behavior?

Here's a minimal example. When I run it, the two print statements show that before and after calling switch_mode_and_capture_request the exposure time value changes.

import time
from collections.abc import Buffer
from copy import deepcopy
from io import BufferedIOBase, BytesIO
from threading import Lock
from typing import Any

from picamera2.encoders import MJPEGEncoder, Quality
from picamera2.outputs.fileoutput import FileOutput
from picamera2.picamera2 import Picamera2


class StreamingOutput(BufferedIOBase):
    """Simulated Buffered IO to write the frames to as they are encoded."""

    def __init__(self) -> None:
        """Initialize the writer."""
        self.frame: bytes | None = None
        self.condition = Lock()

    def write(self, buf: Buffer) -> int:
        """Write the bytes to the frame.

        Arguments:
            buf: The buffer to write

        Returns:
            Number of written bytes

        """
        with self.condition:
            self.frame = BytesIO(buf).read()
            return len(self.frame)


cam_controls: dict[str, Any] = {
    "AeEnable": True,
    "ExposureValue": 0.0,
    "FrameRate": 30.0,
}

picam2 = Picamera2()

output = StreamingOutput()

preview_config = picam2.create_video_configuration(
    main={"size": (2028, 1520), "format": "YUV420"},
    controls=cam_controls,
)

picam2.configure(preview_config)
picam2.start_recording(
    MJPEGEncoder(cam_controls["FrameRate"]),
    FileOutput(output),
    quality=Quality.VERY_HIGH,
)


time.sleep(1)

###########################################
#               Modify exposure time
###########################################

cam_controls.update(
    {"ExposureTime": 1_000_000, "FrameDurationLimits": (0, 1_000_000)},
)
preview_controls = deepcopy(cam_controls)
preview_controls.update({"FrameDurationLimits": (0, 1_000_000)})
picam2.set_controls(preview_controls)

time.sleep(2)

###########################################
#               Take the photo
###########################################

# Create config for high res photo
capture_config = picam2.create_still_configuration(
    raw={"format": "SRGGB12", "size": (4056, 3040)},
    display=None,
    controls=cam_controls,
)
# Stop the encoder to prevent crashes
# See https://forums.raspberrypi.com/viewtopic.php?t=354226
picam2.stop_encoder()


print(picam2.capture_metadata()["ExposureTime"])
request = picam2.switch_mode_and_capture_request(
    capture_config,
)
print(picam2.capture_metadata()["ExposureTime"])

picam2.start_encoder(MJPEGEncoder(), FileOutput(output))

# Release the request
request.release()

[davidplowman]

Hi, thanks for sending the test case. I think the main cause of this may be that you've included the cam_controls with your preview configuration. This means that, whenever your preview_config gets applied (either when you start the camera, or after the mode switch), it reapplies those controls. This includes the framerate of 30, which truncates that 1 second exposure time right back to approximately 33ms.

The other slight catch is that create_video_configuration will, unless you say otherwise, also give you 30fps by default.

So here's what I would do:

1. No need to set those cam_controls with either the preview or capture config. They're all defaults anyway. If you want to set controls before the camera starts, but not have them reapplied when you go back to that mode, then you can use picam2.set_controls() between picam2.configure() and picam2.start().

2. We do need to stop create_video_configuration() from setting you back to 30fps all the time. So when you create that preview config, I would add controls={'FrameDurationLimits': (0, 1_000_000)}.

Here's a further simplified version of what you had:

import time
from picamera2.picamera2 import Picamera2

picam2 = Picamera2()

limits = {'FrameDurationLimits': (0, 1_000_000)}

preview_config = picam2.create_video_configuration(
    main={"size": (2028, 1520), "format": "YUV420"},
    controls=limits
)

picam2.configure(preview_config)
picam2.set_controls({'FrameRate': 30})
picam2.start()

time.sleep(1)

###########################################
#               Modify exposure time
###########################################

picam2.set_controls(limits | {'ExposureTime': 1_000_000})

time.sleep(2)

###########################################
#               Take the photo
###########################################

capture_config = picam2.create_still_configuration(controls=limits)

print(picam2.capture_metadata()["ExposureTime"])
request = picam2.switch_mode_and_capture_request(
    capture_config,
)
print(picam2.capture_metadata()["ExposureTime"])

request.release()

Note that I've put wide frame duration limits on all the modes, so they'll never truncate you back to 30fps or anything. But outside of the preview config, I've set 30fps before starting the camera, so any recording (if I hadn't removed it!) would be at 30fps. Do be aware, however, that at the end of this script, you're still running at exposure time = 1 second, i.e. 1fps.

Hope that helps!

[marsfan]

That helped me out a lot! Turns out it was all from those default values in cam_controls. In my full application (The example I provided is a super-stripped down version of code from this codebase), I simply cleared out the cam_controls dictionary and removed the reading of it for framerate from setting up the MJPEG encoder, and it seems to be working properly now!

Of course, its now messed up some other stuff that assumed that those elements would be in the dictionary, so I'll need to re-work some other parts of the codebase now to read the metadata some other way, but I'm sure I'll figure out how to make that work again.

For reference, if you look at my original example in the original post, there were only two changes:

1. Change the initial declaration of cam_controls to

cam_controls: dict[str, Any] = {
    "FrameDurationLimits": (0, 1_000_000),
}

2. Remove explicitly setting the framerate when setting up the MJPEG encoder:

picam2.start_recording(
    MJPEGEncoder(),
    FileOutput(output),
    quality=Quality.VERY_HIGH,
)

I didn't realize that it was using those to essentially force-reset the camera configuration after switching back.

[marsfan]

@davidplowman As a follow on, I had previously been reading whether or not AE Enabled was enabled and the EV value from that cam_controls dictionary.

I'm trying to see if I can get them from the metadata as well.

• I can't find a metadata property for the EV. Does this exist?
• I do see an AeState that seems to change between 0 and 2, but looking through the PiCamera documentation, I don't see anything about what those numbers mean.

[marsfan]

Oh hang on. Looks like I can read the picam2.camera_controls object.

[marsfan]

No. Forgot that shows the valid range.

Seems sometimes "ExposureValue" appears for the frame's metadata, but it does not always. Not sure why.

[davidplowman]

It's not actually strictly defined when metadata is meant to appear and when it doesn't, so there are definitely some grey areas here. Settings that change spontaneously (e.g. exposure time, analogue gain, colour gains/matrix typically) are reported all the time. Stuff that only the application changes are mostly only reported when the algorithms have "seen" them, but not otherwise.

ExposureValue will fall into this latter category. When you change it, your new value should be acknowledged a few (or several) frames later, at which point you know the algorithm "is doing it". There's certainly an argument for reporting everything all the time, but so far as I know there's no libcamera policy on this.

Other than that, your application would have to remember what it last sent, you are at least guaranteed it won't change under your feet.