Add support for managing appstream catalogs as part of the index#192
Open
joebonrichie wants to merge 6 commits intomainfrom
Open
Add support for managing appstream catalogs as part of the index#192joebonrichie wants to merge 6 commits intomainfrom
joebonrichie wants to merge 6 commits intomainfrom
Conversation
Example xml file:
```xml
<?xml version="1.0" encoding="UTF-8"?>
<PISI>
<AppstreamCatalogs>
<AppstreamCatalog>
<Origin>solus-shannon-main</Origin>
<URI>https://appstream.getsol.us/data/shannon/main/Components-x86_64.xml.gz</URI>
<Icons size="64x64">
<URI>https://appstream.getsol.us/data/shannon/main/icons-64x64.tar.gz</URI>
</Icons>
</AppstreamCatalog>
<AppstreamCatalog>
<Origin>solus-unstable-main</Origin>
<URI>https://appstream.getsol.us/data/unstable/main/Components-x86_64.xml.gz</URI>
<Icons size="64x64">
<URI>https://appstream.getsol.us/data/unstable/main/icons-64x64.tar.gz</URI>
</Icons>
<Icons size="64x64@2">
<URI>https://appstream.getsol.us/data/unstable/main/icons-64x64@2.tar.gz</URI>
</Icons>
</AppstreamCatalog>
</AppstreamCatalogs>
</PISI>
```
Important: the origin must match the origin in the catalog, e.g.
`<components version="1.0" origin="solus-shannon-main"
media_baseurl="https://appstream.getsol.us/media/shannon"
time="20250522T131004">`
joebonrichie
force-pushed
the
appstream-catalogs
branch
from
a50f7ef to
54234e5
Compare
EbonJaeger
requested changes
Apr 29, 2026
Comment on lines
+38
to
+42
| catalog_info = { | ||
| "uri": c.getTagData("URI"), | ||
| "icons_sizes": [x.getAttribute("size") for x in c.tags("Icons")], | ||
| "icon_urls": [x.getTagData("URI") for x in c.tags("Icons")], | ||
| } |
Member
There was a problem hiding this comment.
Should this be a data class?
Comment on lines
+316
to
+317
| if headers_only: | ||
| return fetch |
Member
There was a problem hiding this comment.
I'm not sure about this. It seems like if something needs the fetcher, it should be creating one, and not using this function.
Contributor
Author
There was a problem hiding this comment.
Yeah, it should be easier to use a reusable client, this is a bit of hack on the existing codebase i'll admit
|
|
||
|
|
||
| def add_appstreams(path): | ||
| ctx.ui.info("Adding appstream.xml to index") |
Member
There was a problem hiding this comment.
Suggested change
| ctx.ui.info("Adding appstream.xml to index") | |
| ctx.ui.info(_("Adding appstream.xml to index")) |
Member
There was a problem hiding this comment.
Actually, this line isn't very descriptive; is there anything better we could put here?
Contributor
Author
There was a problem hiding this comment.
Followed the same pattern as add_groups(), add_components(), etc. here. If you can think of something better let's do it for all of the supplemental XML files.
Comment on lines
+362
to
+373
| def nuke_dir_recursive_glob(pattern): | ||
| """Remove a directory recursively matching a glob e.g. rm -fr foo*""" | ||
| import glob | ||
| pattern = f"{pattern}*" | ||
| for path in glob.glob(pattern): | ||
| try: | ||
| if os.path.isdir(path) and not os.path.islink(path): | ||
| shutil.rmtree(path) | ||
| else: | ||
| os.remove(path) | ||
| except Exception as e: | ||
| print(f"Error removing {path}: {e}") |
Member
There was a problem hiding this comment.
Do we really need a whole function for this?
Contributor
Author
There was a problem hiding this comment.
Seemed cleaner than having it inline especially as other similar functions live here. Having a cleaner way to remove obsolete catalogs seems more ideal.
joebonrichie
force-pushed
the
appstream-catalogs
branch
from
54234e5 to
93d9a0c
Compare
joebonrichie
force-pushed
the
appstream-catalogs
branch
from
93d9a0c to
4e82e52
Compare
Support downloading and extracting appstream catalogs and their associated icons marked in the index when updating repositories Check for obsolete catalogs when updating the repositories
joebonrichie
force-pushed
the
appstream-catalogs
branch
from
4e82e52 to
59e8d8f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Why
We want up-to-date appstream catalog information as some applications contain changelog information in their appstream metadata. This changelog information is then shown in front-ends such as gnome-software. As is it currently, appstream-catalog is downloaded as part of a separate package, this then means that gnome-software will show out of date changelog information for packages. e.g. an update to gnome-clocks 49.0 exists and the changelog for this release is included an update for appstream-catalog. However, as the latest appstream-catalog is not yet installed gnome-software will either show out of date changelog information or simply fall back to the generic summary message used for the update. By updating the appstream-catalog as part of the eopkg index we can then ensure we always have the most up-to-date changelog information for available updates.
Example
Before up-to-date appstream catalog is installed:
After up-to-date appstream catalog is installed:
How
A supplemental index file can be provided (
appstream.xml) that is picked up similarly in nature to other supplemental files e.g.components.xml,groups.xml.An example of an appstream.xml file is:
Multiple appstream catalogs can be specified.
When the repository is indexed the
</AppstreamCatalog>tag(s) is included in theeopkg-index.xmlfile.The origin should match the origin in the
Components-x86_64.xml.gzfile.When an
</AppstreamCatalog>is included in the eopkg index, eopkg will then download the appstream catalog and it's icons to/var/lib/swcatalog/xml/if the catalog is not installed. If the catalog is installed then eopkg will check against the Last-Modified header to see if newer tarballs exist on the remote then download and install them if necessary.For the example
appstream.xmlfile above, eopkg will install to the following paths:/var/lib/swcatalog/xml/solus-unstable-main.xml.gz/var/lib/swcatalog/xml/solus-unstable-main.xml.gz.eopkg/var/lib/swcatalog/icons/solus-unstable-main/128x128//var/lib/swcatalog/icons/solus-unstable-main/64x64//var/lib/swcatalog/icons/solus-unstable-main/64x64@2/Eopkg will place a tag on the catalog so it knows that appstream catalog is managed by itself (in this case
/var/lib/swcatalog/xml/solus-unstable-main.xml.gz.eopkg) . If the index no longer contains an appstream catalog matching the tag, eopkg will then nuke those paths as an obsolete catalog when the index is updated.Test Plan
appstream.xmlto local repo</AppstreamCatalog>exists in theeopkg.index.xmlfileappstream.xmlfrom local repo</AppstreamCatalog>no longer exists ineopkg-index.xmlfileTODO: been a while since i've looked at this code but i think i wasn't particularly happy with how obsolete catalogs were managed