[Libosinfo] [PATCH osinfo-db 0/2] <tree arch="unknown"> fallback

[Cole Robinson]

On 3/1/19 3:19 AM, Fabiano Fidêncio wrote:
> On Fri, Mar 1, 2019 at 12:48 AM Cole Robinson <crobinso at redhat.com> wrote:
>>
>> <tree> elements in the OS DB serve two broad purposes:
>>
>> 1) track public URLs, so apps can easily look up an install URL and
>>    associated metadata like relative initrd/kernel paths
>> 2) track <treeinfo> matching data, so users can use guess_os_by_tree
>>    with their own URL, and get an OS object back
>>
>> Every <tree> needs an explicit @arch value associated. This makes
>> thoroughly covering case #2 difficult, because it requires enumerating
>> ever supported OS arch in a <tree> block. This is useful to do for
>> OS that have public URLs, because we want to enumerate those anyways.
>> But for distros without public URLs (RHEL), getting coverage for #2
>> requires adding a bunch of <tree> stubs for each supported @arch for
>> not much gain.
>>
>> Additionally consider Fedora, which has primary architectures
>> (aarch64, armv7hl, x86_64) and secondary architectures like ppc64le.
>> The latter aren't necessarily considered official releases, so it's
>> debatable whether to enumerate their install URLs in the fedora OS
>> entries. However it's still useful to detect the URL as 'fedora',
>> so virt-install attempts with a manually passed URL get proper defaults
>> etc.
>>
>> This series makes it easier to add a catchall <tree> entry that is
>> arch independent by specifying arch="unknown", and demonstrates using
>> it to get treeinfo detection coverage for rhel5.
>>
>> One point: I don't think this arch=unknown will cause problems for API
>> users. For case #1, users won't be interacting with this <tree>
>> because it doesn't have any URL listed. For case #2, the user will
>> already have to call osinfo_tree_create_from_location which will
>> give a tree with relevant arch/kernel/initrd data parsed out of
>> the treeinfo, and they should use that OsinfoTree object and not the
>> OsinfoTree returned by guess_os_by_tree
> 
> There's a small catch though. We have to adapt the
> osinfo_db_guess_os_from_tree() code to deal with the arch=unknown as
> what it currently does is:
> ```
>  801             if (match_regex(os_family, tree_family) &&
>  802                 match_regex(os_variant, tree_variant) &&
>  803                 match_regex(os_version, tree_version) &&
>  804                 match_regex(os_arch, tree_arch)) {
>  805                 ret = os;
>  806                 if (matched_tree != NULL)
>  807                     *matched_tree = os_tree;
>  808                 break;
>  809             }
> ```
> 

That's comparing two ./tree/treeinfo/arch values, not ./tree/@arch.
Since in my RHEL5 examples there's no ./tree/treeinfo/arch, the
comparison is effectively arch independent. It behaves similar to when
<variant> is missing for example

> A second comment ... *upstream* does not have the URLs, indeed and
> this solution fits well there.
> Downstream (not only RHEL) any other distro may have entries for each
> one of their trees distributed as an additional package. For instance,
> let's say we internally create something like: osinfo-db-rhel which
> would have to contain all the trees with URLs ... we'd have to take
> into consideration the "unknown" and avoid matching it against the
> "known" ones. So, more logic to be adjusted in the
> osinfo_db_guess_os_from_tree().
> 

Good point. Maybe some XML annotation that explicitly marks the <tree>
or <media> as a fallback or catchall, and we check against that last. Or
just have arch="unknown" imply that.

> Considering everything said about the trees ... we still do face the
> very same issues with the ISOs and we need to also decide what to do
> with those.
>
Indeed. <media> fallback matching is a little worse off than <tree>,
because media_create_from_location won't detect media arch, it depends
on guess_os_from_media to fill that value in with the returned
OsinfoMedia object. But with this style of fallback matching there won't
be any non-unknown returned arch value. Probably a minor issue in
practice but I wanted to point it out.

- Cole

> I guess the main question here is how we can have a generic info that
> can match every release of an OS ... but also have specialised entries
> that *should* be taken into consideration even if they match an
> "unknown" entry (before matching the right one).
> 
> Let's keep the discussion (and I'll re-post the tree series for RHEL-6
> and RHEL-7 based on what we decide here).