Skip to content

Feature/add image import#634

Open
Andre-85 wants to merge 1 commit intocontainers:mainfrom
Andre-85:feature/add_image_import
Open

Feature/add image import#634
Andre-85 wants to merge 1 commit intocontainers:mainfrom
Andre-85:feature/add_image_import

Conversation

@Andre-85
Copy link
Copy Markdown

@Andre-85 Andre-85 commented Mar 5, 2026

Hi @ALL,
while using podman-py i realized that there is no python-binding for podman import, so i created one by .myself.

I works straight forward:

import podman
c = podman.PodmanClient()
c.images.import_image(<tarfile name>, reference=<like in the podman import command>)

I tested it successfully on Ubuntu 24.04 with podman 4.9.3 and on Ubuntu 26.04 snapshot 3 (snapshot 4 does not work on qemu for some reason) and podman 5.7.0.

What do you think?

Greetings,
André

@Andre-85 Andre-85 force-pushed the feature/add_image_import branch from 0a359b2 to aace063 Compare March 6, 2026 08:19
Copy link
Copy Markdown
Member

@Honny1 Honny1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add tests for this. Ideally integration tests, since it doesn’t work on your machine with the latest version of Podman.

@Andre-85
Copy link
Copy Markdown
Author

Andre-85 commented Mar 6, 2026

Please add tests for this. Ideally integration tests, since it doesn’t work on your machine with the latest version of Podman.

Yes i will do in the next days.

Greetings,
André

Comment thread podman/domain/images_manager.py Outdated
"""Import a tarball as an image (equivalent of 'podman import').

Args:
source: Path to a tarball (str) or a file-like object opened in binary mode.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This arg does not exist.

Comment thread podman/domain/images_manager.py Outdated
APIError: when service returns an error.
"""
# Check that exactly one of the data or file_path is provided
if not data and not file_path:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would use is None check.

Comment thread podman/tests/unit/test_imagesmanager.py Outdated
self.client.images.import_image(b'data', b'file_path')

with self.assertRaises(PodmanError):
self.client.images.import_image(data=b'data', file_path=b'file_path')
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is file_path a bytes literal?

Comment thread podman/domain/images_manager.py Outdated
if file_path:
# Convert to Path if file_path is a string
file_path_object = Path(file_path)
post_data = file_path_object.read_bytes() # Read the tarball file as bytes
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would stream this data instead of reading a multiple GB file into memory.

@Honny1
Copy link
Copy Markdown
Member

Honny1 commented Mar 9, 2026

Also, please squash your changes into one commit.

@Andre-85
Copy link
Copy Markdown
Author

Also, please squash your changes into one commit.

Yes, will do and i will apply your suggestions.

@Andre-85 Andre-85 force-pushed the feature/add_image_import branch 2 times, most recently from 71692e1 to eae68f5 Compare March 10, 2026 10:23
@Andre-85 Andre-85 requested a review from Honny1 March 10, 2026 10:52
@Andre-85
Copy link
Copy Markdown
Author

Also, please squash your changes into one commit.

Yes, will do and i will apply your suggestions.

I applied the requested changes, added extra tests for setting message and changes and squashed all changes in one commit.

Copy link
Copy Markdown
Member

@Honny1 Honny1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ruff (legacy alias)......................................................Failed
- hook id: ruff
- exit code: 1

UP035 `typing.List` is deprecated, use `list` instead
  --> podman/domain/images_manager.py:9:1
   |
 7 | import os
 8 | import urllib.parse
 9 | from typing import Any, Literal, Optional, Union, List
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
10 | from collections.abc import Iterator, Mapping, Generator
11 | from pathlib import Path
   |

UP006 Use `list` instead of `List` for type annotation
   --> podman/domain/images_manager.py:175:27
    |
173 |         reference: Optional[str] = None,
174 |         message: Optional[str] = None,
175 |         changes: Optional[List[str]] = None,
    |                           ^^^^
176 |     ) -> "Image":
177 |         """Import a tarball as an image (equivalent of 'podman import').
    |
help: Replace with `list`

F841 Local variable `tar_path` is assigned to but never used
   --> podman/tests/integration/test_images.py:327:13
    |
325 |             # Pack the testfile with the test folder in a tar buffer
326 |             tar_buffer = io.BytesIO()
327 |             tar_path = os.path.join(tmpdir, "test.tar.gz")
    |             ^^^^^^^^
328 |             with tarfile.open(fileobj=tar_buffer, mode="w:gz") as tar:
329 |                 tar.add(base_dir, arcname="test")
    |
help: Remove assignment to unused variable `tar_path`

Found 3 errors.
No fixes available (2 hidden fixes can be enabled with the `--unsafe-fixes` option).

@Andre-85 Andre-85 force-pushed the feature/add_image_import branch 2 times, most recently from 15fa761 to 0cab523 Compare March 10, 2026 20:58
@Andre-85 Andre-85 requested a review from Honny1 March 10, 2026 21:15
@Andre-85
Copy link
Copy Markdown
Author

ruff (legacy alias)......................................................Failed
- hook id: ruff
- exit code: 1

UP035 `typing.List` is deprecated, use `list` instead
  --> podman/domain/images_manager.py:9:1
   |
 7 | import os
 8 | import urllib.parse
 9 | from typing import Any, Literal, Optional, Union, List
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
10 | from collections.abc import Iterator, Mapping, Generator
11 | from pathlib import Path
   |

UP006 Use `list` instead of `List` for type annotation
   --> podman/domain/images_manager.py:175:27
    |
173 |         reference: Optional[str] = None,
174 |         message: Optional[str] = None,
175 |         changes: Optional[List[str]] = None,
    |                           ^^^^
176 |     ) -> "Image":
177 |         """Import a tarball as an image (equivalent of 'podman import').
    |
help: Replace with `list`

F841 Local variable `tar_path` is assigned to but never used
   --> podman/tests/integration/test_images.py:327:13
    |
325 |             # Pack the testfile with the test folder in a tar buffer
326 |             tar_buffer = io.BytesIO()
327 |             tar_path = os.path.join(tmpdir, "test.tar.gz")
    |             ^^^^^^^^
328 |             with tarfile.open(fileobj=tar_buffer, mode="w:gz") as tar:
329 |                 tar.add(base_dir, arcname="test")
    |
help: Remove assignment to unused variable `tar_path`

Found 3 errors.
No fixes available (2 hidden fixes can be enabled with the `--unsafe-fixes` option).

I fixed the issues.

Comment thread podman/domain/images_manager.py Outdated
"""Import a tarball as an image (equivalent of 'podman import').

Args:
file_path: Path to a tarball (str) or a file-like object opened in binary mode.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think description doesn't match file_path: Optional[os.PathLike] = None.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fixed it.

Comment thread podman/domain/images_manager.py Outdated
response.raise_for_status()

body = response.json()
image_id = body.get("Id") or body.get("id")
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if [Ii]d is missing?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup, this should be handled for failure of getting Id

Copy link
Copy Markdown
Author

@Andre-85 Andre-85 Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found out the true field name is "Id" so i will leave the other one out.

I guess i found a similar error handling as requested in the load function:

        response.raise_for_status()  # Catch any errors before proceeding
  
          def _generator(body: dict) -> Generator[Image, None, None]:
              # Iterate and yield images from response body
              for item in body["Names"]:
                  yield self.get(item)
  
          # Pass the response body to the generator
          return _generator(response.json())

But being honest, I don't get what is happening here...

@Honny1
Copy link
Copy Markdown
Member

Honny1 commented Mar 11, 2026

PTAL @inknos

@inknos inknos self-requested a review March 11, 2026 13:00
Comment thread podman/domain/images_manager.py Outdated
Comment on lines +171 to +175
data: Optional[bytes] = None,
file_path: Optional[os.PathLike] = None,
reference: Optional[str] = None,
message: Optional[str] = None,
changes: Optional[builtins.list[str]] = None,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'd like to see only the required args as named, the others being kwargs. It's an arbitrary approach, but the library is designed this way, so I would follow this design pattern.

def import_image(
    self,     
    data: Optional[bytes] = None,
    file_path: Optional[os.PathLike] = None,
    **kwargs,

Kwargs being: reference, message, changes, and url, which is missing. Do you plan to implement it? if not, we'll have to document it with TODO and reference a new issue in the docstring.

Copy link
Copy Markdown
Author

@Andre-85 Andre-85 Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i fixed the style. reference, message and changes are implemented and tested (in the integration test).

The url parameter i did not implement (yet). Would be a solution with with urllib.request.urlopen(source) as response: ... ok for you?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or alternate with requests:

  with requests.get('https://httpbin.org/get', stream=True) as r:
    for chunk in r.iter_content(chunk_size=8192):
        ....

Comment on lines +203 to +209
params = {}
if reference:
params["reference"] = reference
if message:
params["message"] = message
if changes:
params["changes"] = changes # requests sends repeated keys as a list
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following the comment above, this can be simplified with the same logic of other functions that do kwargs.get()

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i fixed it

@Andre-85
Copy link
Copy Markdown
Author

@Honny1
@inknos

I will do the remaining fixes in the next days. After resolving all i will squeeze them in one commit.

@Andre-85 Andre-85 force-pushed the feature/add_image_import branch 3 times, most recently from 165cfdc to b235fe9 Compare March 15, 2026 20:52
@Andre-85
Copy link
Copy Markdown
Author

Hi,
I implemented the request changes and I implemented the missing url feature including unit and integration tests.

Greetings,
André

@Andre-85 Andre-85 requested a review from Honny1 March 16, 2026 08:18
Comment thread podman/domain/images_manager.py Outdated
params["url"] = url

# Get either from file or from raw data
post_data_context = Path(file_path).open("rb") if file_path else io.BytesIO(data)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if io.BytesIO(None)?

Comment thread podman/tests/unit/test_imagesmanager.py Outdated
with patch("pathlib.Path.open", return_value=io.BytesIO(b"mock tarball data")):
mock.post(
tests.LIBPOD_URL + "/images/import",
json={"Id": "quay.io/fedora:latest"},
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will the actual import return an image tag or an image ID?

@Andre-85 Andre-85 requested a review from Honny1 March 19, 2026 17:45
@Andre-85
Copy link
Copy Markdown
Author

Hi @ALL,
I implemented the requests.

To the requests and the fixes:

  1. io.BytesIO(None): Instead of returning a empty temporary file, i directly None which is the default value of the optional data kwarg parameter
  2. Mock for images/import: Yes you were right, and id (or better digest) is returned.

Greetings,
André

Comment thread podman/tests/integration/test_images.py Outdated
Comment thread podman/tests/integration/test_images.py
Comment thread podman/domain/images_manager.py Outdated
@Andre-85 Andre-85 force-pushed the feature/add_image_import branch from 27fa6b6 to bba954e Compare March 26, 2026 08:14
@Andre-85 Andre-85 requested a review from Honny1 March 26, 2026 08:42
Copy link
Copy Markdown
Member

@Honny1 Honny1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have last questions about changes.

"/images/import",
params=params,
data=post_data,
headers={"Content-Type": "application/x-tar"},
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this Header correct when the input is a URL?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did some investigation:

I assume that podman --remote is using the API correctly.

Since importing images and using an alternate podman socket is not allow sametime (some error like "url and r parameter is not allowed sametime" is shown), put a MITM with socket and started the podman system service on a different socket:

podman system service unix:///tmp/podman.sock --time=0
socat -v UNIX-LISTEN:/run/user/1000/podman/podman.sock UNIX-CONNECT:/tmp/podman.sock
podman --remote import ...

The result for an empty file loaded from file_path is:

> 2026/03/27 09:59:38.000477279  length=78 from=0 to=77
GET /v4.9.3/libpod/_ping HTTP/1.1\r
Host: d\r
User-Agent: Go-http-client/1.1\r
\r
< 2026/03/27 09:59:38.000477854  length=380 from=0 to=379
HTTP/1.1 200 OK\r
Api-Version: 1.41\r
Builder-Version: \r
Buildkit-Version: \r
Cache-Control: no-cache\r
Docker-Experimental: true\r
Libpod-Api-Version: 4.9.3\r
Libpod-Buildah-Version: 1.33.7\r
Ostype: linux\r
Pragma: no-cache\r
Server: Libpod/4.9.3 (linux)\r
X-Reference-Id: 0xc000708010\r
Date: Fri, 27 Mar 2026 08:59:38 GMT\r
Content-Length: 2\r
Content-Type: text/plain; charset=utf-8\r
\r
OK

For an non-empty file loaded from file_path:

> 2026/03/27 10:03:13.000103092  length=78 from=0 to=77
GET /v4.9.3/libpod/_ping HTTP/1.1\r
Host: d\r
User-Agent: Go-http-client/1.1\r
\r
< 2026/03/27 10:03:13.000103678  length=380 from=0 to=379
HTTP/1.1 200 OK\r
Api-Version: 1.41\r
Builder-Version: \r
Buildkit-Version: \r
Cache-Control: no-cache\r
Docker-Experimental: true\r
Libpod-Api-Version: 4.9.3\r
Libpod-Buildah-Version: 1.33.7\r
Ostype: linux\r
Pragma: no-cache\r
Server: Libpod/4.9.3 (linux)\r
X-Reference-Id: 0xc000708010\r
Date: Fri, 27 Mar 2026 09:03:13 GMT\r
Content-Length: 2\r
Content-Type: text/plain; charset=utf-8\r
\r
OK

For an non-empty file loaded from url:

> 2026/03/27 09:53:33.000864558  length=78 from=0 to=77
GET /v4.9.3/libpod/_ping HTTP/1.1\r
Host: d\r
User-Agent: Go-http-client/1.1\r
\r
< 2026/03/27 09:53:33.000865215  length=380 from=0 to=379
HTTP/1.1 200 OK\r
Api-Version: 1.41\r
Builder-Version: \r
Buildkit-Version: \r
Cache-Control: no-cache\r
Docker-Experimental: true\r
Libpod-Api-Version: 4.9.3\r
Libpod-Buildah-Version: 1.33.7\r
Ostype: linux\r
Pragma: no-cache\r
Server: Libpod/4.9.3 (linux)\r
X-Reference-Id: 0xc000392028\r
Date: Fri, 27 Mar 2026 08:53:33 GMT\r
Content-Length: 2\r
Content-Type: text/plain; charset=utf-8\r
\r
OK

So in all cases the Content-Type is set to text/plain with looks odd, i would say setting in all cases application/x-tar like I did looks more resonable or?

What do you think?

Greetings,
André

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems odd. The first thing I noticed is an older version of Podman (4.9.3). Can you please test it with version 5.8?

# Pass the response body to the generator
return _generator(response.json())

def import_image(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nonblocking: I'm not sure if podman-py is a drop-in replacement for docker-py. Is it? @inknos

If so, we should probably match docker-py.

Copy link
Copy Markdown
Author

@Andre-85 Andre-85 Mar 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi,
I did some comparison to docker-py.
First i saw that docker-py is using a different endpoint, it uses images/create. But from the description (Link: https://docs.docker.com/reference/api/engine/version/v1.54/#tag/Image/operation/ImageCreate) i would say that looks like misuse of this endpoint, because the descriptions are all pointing to oci images and not to raw tar files.

I also did a comparison with command line podman (podman --remote --debug import import.tar). I can see here that command line podman is also using the equivalent of images/import like my patch. In general I would say that for podman-py it is more important to mimic the behavior of command line podman than the behavior of docker-py.

Further I think that their design for the import_image functions have serious problems (Link: https://github.com/docker/docker-py/blob/main/tests/integration/api_image_test.py#L161) finding out for the src arg, what kind of source it is (file/url or raw bytes). So i think the approach with kwargs data, file_path and url is better and fits more to the load function in podman-py which uses already data.

But i think also providing the convenience wrappers import_image_from_url, import_image_from_data and import_image_from_file like docker-py does should be simple to do.

What do you think?

Greetings,
André

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, Podman tries to be a replacement of Docker, so podman-py should be able to replace docker-py. On one hand, I think it doesn't matter if it is called the libpod API or the compat API of Podman if is supported only for full replacement, but it would mean that with podman-py you cannot use Docker. I am not sure which way this compatibility should work.

cc @inknos

So definitely, I think convenience wrappers are a good idea, but for another PR.

Comment thread podman/domain/images_manager.py
@Andre-85 Andre-85 force-pushed the feature/add_image_import branch from 5b67896 to f65fb03 Compare March 27, 2026 10:38
@Andre-85
Copy link
Copy Markdown
Author

I have made comments about docker-py compatibility and about the Content-Type (see above). The data=b"" corner case I fixed

@Andre-85 Andre-85 requested a review from Honny1 March 27, 2026 12:10
Comment thread podman/domain/images_manager.py Outdated
Comment on lines +197 to +203
if data is None and file_path is None and url is None:
raise PodmanError("The 'data' or 'file_path' or 'url' parameter should be set.")

if (data and file_path) or (url and data) or (url and file_path):
raise PodmanError(
"Only one parameter should be set from 'data', 'file_path' and 'url' parameters."
)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if data is None and file_path is None and url is None:
raise PodmanError("The 'data' or 'file_path' or 'url' parameter should be set.")
if (data and file_path) or (url and data) or (url and file_path):
raise PodmanError(
"Only one parameter should be set from 'data', 'file_path' and 'url' parameters."
)
if sum(x is not None for x in (data, file_path, url)) > 1:
raise PodmanError("Only one parameter should be set from 'data', 'file_path' and 'url' parameters.")

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Guess you did a minor mistake here: You change proposal covers only the more than one parameter set, but not the none of the parameters set. So i change check from >1 to !=1 and i adapted the error message a bit

self.assertEqual(actual[1]["linkTarget"], "/test/foobar")

# Clean up
image.remove(force=True)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't forget about the container.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In all integration tests.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i was not aware that deleting the image the containers are based on is not deleting them automatically. I will fix it

# Create test folder with test file
base_dir = os.path.join(tmpdir, "test")
os.mkdir(base_dir)
open(os.path.join(base_dir, "foobar"), "w").close()
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
open(os.path.join(base_dir, "foobar"), "w").close()
Path(base_dir, "foobar").touch()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In all integration tests.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought since you are not using pathlib at all within your integration tests, I should not use it for a single statement and do a mixture of pathlib / os module file handling?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this looks cleaner, but it is not blocking:

base_dir = Path(tmpdir) / "test"
base_dir.mkdir(parents=True, exist_ok=True)
(base_dir / "foobar").touch()

# or

(Path(tmpdir) / "test").mkdir(parents=True, exist_ok=True)
(Path(tmpdir) / "test" / "foobar").touch()

@Andre-85
Copy link
Copy Markdown
Author

Andre-85 commented Apr 7, 2026

Ok, i will do the requested changes the next days

@Andre-85 Andre-85 force-pushed the feature/add_image_import branch from 3c08553 to c7171fc Compare April 8, 2026 09:44
@Andre-85
Copy link
Copy Markdown
Author

Andre-85 commented Apr 8, 2026

I did the requested changes and/or modified them a bit, in case of using pathlib i omitted it, see above. I did no further adding of functions for docker-py compat, since you said that this should not block here. Perhaps we should do this in an own pull-request.

Greetings,
André

for importing raw tar files. This corresponds "podman image import <path> [reference]" on the command line

Signed-off-by: Andre Wagner <wagnerandre85@aol.de>
@Andre-85 Andre-85 force-pushed the feature/add_image_import branch from 0f4140d to 2db0ef4 Compare April 9, 2026 08:39
@Andre-85
Copy link
Copy Markdown
Author

Andre-85 commented Apr 9, 2026

Just did an rewrite of a lambda to make a function to pass "pre-commit"

@Honny1 Honny1 requested a review from inknos April 13, 2026 13:04
Copy link
Copy Markdown
Member

@Honny1 Honny1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code LGTM. Just a non-blocking comment. Also, I'm not sure how strictly we should enforce compatibility between Podman and Docker.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants