.. _suite-relationships:

===================
Suite relationships
===================

Goals
=====

This blueprint defines a structured way to record relationships between
collections, starting with :collection:`debian:suite` collections.

Requirements
============

  * Record relationships from one collection to other collections.
    Possible relationships with its cardinality:

    * ``forked_from``: the suite this suite was forked from (0..1)
    * ``based_on``: the original base suite this one derives from (0..1)
    * ``requires``: suites whose APT sources are required to use/build/test
      this suite (0..N, ordered)
    * ``targeting``: the suite where packages in this suite are expected to land
      (0..1)
    * ``default_qa_results``: the ``debian:qa-results`` collection
      to use by default for this suite (0..1)
  * Work across workspace inheritance
  * Allow relationships from suites to non-suite collections (e.g. to define
    the default :collection:`debian:qa-results` collection)
  * Remove (or make optional, while preserving functionality) the need to specify
    suites manually in places such as ``qa_suite`` or ``reference_qa_results``.

For ``requires``, the relationship is ordered. When generating the
``sources.list`` or equivalent, entries are emitted in ascending ``position``
order.

Use cases
=========

Support transitions
-------------------

One of the goals of collection relationships is to support transitions.

Once the relationships between collections are added, Debusine can infer
automatically which is the ``suite``, the ``qa_suite`` or the
``regression_tracking_qa_results``. This will help supporting transitions.

Simplify workflows
------------------

Initially all the workflow variables will remain the same. But, if they are
not set, when possible, they will be inferred from the relationships. At a later
stage, the variables could be removed (if left there, they can be used to
override the relationships)

  * ``DebianPipelineWorkflow``
      * look up the ``primary_suite`` from ``(vendor, codename)``
      * define ``suite_under_test = suite if not None else primary_suite``
        (in other words: ``DebianPipelineWorkflow.suite`` if the user
        has specified it, else it is the ``primary_suite``)
      * Infer:
        * ``qa_suite``: ``suite_under_test.targeting if not None else primary_suite``
        * ``regression_tracking_qa_results``: ``qa_suite.suite_default_qa_results``
        * ``check_installability_suite``: same as ``qa_suite``

      * Use ``suite_under_test.requires`` when building ``extra_repositories``

Auto document / simplify personal repositories
----------------------------------------------

This is a particular case of **Simplify workflows** use case.

Debusine users can create a personal workspace and create ``debian:suite``
in there to publish personal repositories.

At the moment, they are isolated from other suites. Adding suite relationships will
help testing by avoiding the need to define duplicated values in the workflow.
It will also help that if a repository changes from requiring ``forky`` to
requiring ``testing`` it will be a single change in the relationships instead
of having to tweak different variables.

This view should indicate other APT sources that are required to use suites
in the archive, according to ``requires``:

  * URL: ``https://<debian-archive-host>/<scope>/<workspace>/``
  * URL example: ``https://deb.debusine.debian.net/debian/r-cjwatson-debusine/``

The ``SuiteRootView`` (``/<scope>/<workspace>/dists/<suite>/``) will include
the APT configuration using ``requires``.


Lookups
=======

No changes in lookups are required.

Implementation plan
===================

Add data model
--------------


.. code-block:: python

    class CollectionRelation(models.Model):
        """Model relations between collections."""

        objects: CollectionRelationManager = CollectionRelationManager()

        Types: TypeAlias = _CollectionRelationTypes

        source = models.ForeignKey(
            Collection, on_delete=models.CASCADE, related_name="relations"
        )

        #: to implement constraints based on category
        source_category = models.CharField(max_length=255)

        target = models.ForeignKey(
            Collection, on_delete=models.CASCADE, related_name="targeted_by"
        )
        #: to implement constraints based on category
        target_category = models.CharField(max_length=255)

        workspace = models.ForeignKey(Workspace, on_delete=models.CASCADE)

        type = models.CharField(max_length=32, choices=_CollectionRelationTypes.choices)

        position = models.PositiveIntegerField(null=True, blank=True)

        # Include fields to record who (token, it could be a worker via API)
        # and when the relation was created or modified.

        class Meta(TypedModelMeta):
            base_manager_name = "objects"

            constraints = [
                # Enforce that `requires` relations have a position, others not
                models.CheckConstraint(
                    name="%(app_label)s_%(class)s_position",
                    check=(
                        Q(
                            type=_CollectionRelationTypes.SUITE_REQUIRES,
                            position__isnull=False,
                        )
                        | Q(
                            type__in=[
                                _CollectionRelationTypes.SUITE_FORKED_FROM,
                                _CollectionRelationTypes.SUITE_BASED_ON,
                                _CollectionRelationTypes.SUITE_TARGETING,
                                _CollectionRelationTypes.SUITE_QA_RESULTS,
                            ],
                            position__isnull=True,
                        )
                    ),
                ),
                models.UniqueConstraint(
                    name="%(app_label)s_%(class)s_suite_requires_no_duplicate_position",
                    fields=["source", "position"],
                    condition=Q(type=_CollectionRelationTypes.SUITE_REQUIRES),
                ),
                models.UniqueConstraint(
                    name="%(app_label)s_%(class)s_cardinality_checks_max_1",
                    fields=["source", "type"],
                    condition=(
                        Q(type=_CollectionRelationTypes.SUITE_FORKED_FROM)
                        | Q(type=_CollectionRelationTypes.SUITE_BASED_ON)
                        | Q(type=_CollectionRelationTypes.SUITE_TARGETING)
                        | Q(type=_CollectionRelationTypes.SUITE_QA_RESULTS)
                    ),
                ),
                models.CheckConstraint(
                    name="%(app_label)s_%(class)s_categories_correct",
                    check=(
                        # SUITE_QA_RESULTS: collection must be a suite,
                        # target must be qa-results
                        Q(
                            type=_CollectionRelationTypes.SUITE_QA_RESULTS,
                            source_category=CollectionCategory.SUITE,
                            target_category=CollectionCategory.SUITE_QA_RESULTS,
                        )
                        # SUITE_FORKED_FROM: both must be suites
                        | Q(
                            type=_CollectionRelationTypes.SUITE_FORKED_FROM,
                            source_category=CollectionCategory.SUITE,
                            target_category=CollectionCategory.SUITE,
                        )
                        # SUITE_REQUIRES: both must be suites
                        | Q(
                            type=_CollectionRelationTypes.SUITE_REQUIRES,
                            source_category=CollectionCategory.SUITE,
                            target_category=CollectionCategory.SUITE,
                        )
                        # SUITE_TARGETING: both must be suites
                        | Q(
                            type=_CollectionRelationTypes.SUITE_TARGETING,
                            source_category=CollectionCategory.SUITE,
                            target_category=CollectionCategory.SUITE,
                        )
                    ),
                ),
                models.CheckConstraint(
                    name="%(app_label)s_%(class)s_source_no_self_referenced",
                    check=~Q(source_id=F("target_id")),
                ),
            ]

CRUD: add CLI support
----------------------

The API will be defined at a later point, following Debusine endpoint
conventions.

CLI commands follow the existing Debusine ``object verb`` style. Since
relationships are between collections, the object is ``collection relation``.

List relations
~~~~~~~~~~~~~~

.. code-block:: shell

    $ debusine collection relation list --workspace WORKSPACE \
        --from NAME@CATEGORY --to NAME@CATEGORY --type TYPE

The output should include, for each relation: source (``from``), target
(``to``), ``type``, and ``position`` (when applicable). Ordered by ``from``,
``to``, ``type`` and ``position``.

The optional parameters ``--from``, ``--to`` and ``--type`` act like a filter.

Edit relation targets
~~~~~~~~~~~~~~~~~~~~~~

This command edits the target list for a given source collection and relation
type. It mimics the ``workspace inheritance`` interface.

.. code-block:: shell

   $ debusine collection relation --workspace WORKSPACE --yaml \
       FROM_NAME@CATEGORY TYPE \
       --append [COLLECTION [COLLECTION...]] \
       --prepend [COLLECTION [COLLECTION...]] \
       --remove [COLLECTION [COLLECTION...]] \
       --set [COLLECTION [COLLECTION...]] \
       --edit

Parameters:

  * ``FROM-NAME@CATEGORY``: source collection whose outgoing relations will be
    edited
  * ``TYPE``: relation type to edit (``fork_parent``, ``requires``,
    ``goal``, ``default_qa_results``)
  * `--append``: add the specified collection(s) to the end of the target list
  * `--prepend``: add the specified collection(s) to the beginning of the target
    list
  * `--remove``: remove the specified collection(s) from the target list
  * `--set``: replace the complete target list with the specified collections
  * `--edit``: edit the complete target list using ``$EDITOR`` (only option
    that does not accept a COLLECTION argument)

The command edits the set of outgoing relations for the specified
``(from, type)`` pair.

For relation types with multiple targets (such as ``requires``), the order of
the target list is significant and defines the ordering of relations.

For relation types with cardinality ``0..1`` (such as ``fork_parent``,
``goal``, and ``default_qa_results``), the list can contain at most one
target. If more than one target is provided, the operation fails (API is called,
API returns a suitable error and client displays the error).

For ``edit``, the editor is opened with one target collection per line,
written as ``NAME@CATEGORY``. For example:

.. code-block:: yaml

  # Here you can edit the targets of the relation list.
  #
  # This is a YAML list of collection identifiers written as NAME@CATEGORY.
  # Each entry represents a target collection for RELATION_TYPE.
  #
  # The order of entries is significant for ordered relation types
  # (such as "requires").
  #
  # Lines starting with '#' are ignored.

   - trixie@debian:suite
   - trixie-security@debian:suite

The user may reorder lines, remove lines, or add new target collections.

For example, replacing ``trixie-security@debian:suite`` with
``trixie-proposed-updates@debian:suite`` and placing it before
``trixie@debian:suite``:

.. code-block:: yaml

   # [INTRODUCTORY COMMENT]

   - trixie-proposed-updates@debian:suite
   - trixie@debian:suite

Each entry in the YAML sequence represents a target collection for the
specified ``(FROM, TYPE)`` relation set.

When the editor exits, the server atomically updates the relations so that the
targets and their order exactly match the provided list. Relations that no longer
appear in the list are deleted, and new relations are created for newly added
targets.

Note that empty list removes all relations of the specified type:

.. code-block:: yaml

  # [ INTRODUCTORY COMMENT ]

  []

An empty file is treated the same way: deletes all relations.

Display and manage collection relationships on the Web
------------------------------------------------------

List relations of a collection
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  * URL structure: ``/<scope>/<wname>/collection/<source_category>/<source_name>/relation/?relation_type=REL_TYPE``
  * URL example: ``/debian/base/collection/debian:suite/trixie/relation/?relation_type=goal``

The view will contain (if no ``relation_type`` query parameter is included):

  * Title: ``Relations of <source_name>@<source_category>``

With a ``relation_type`` query parameter:

  * Title: ``Relations of type "REL_TYPE" of <source_name>@<source_category>``

List with the following columns:

  * Type of relation
  * Target: target collection
  * Position
  * Details. With a link on each relation to display, on demand, creation and
    modification metadata (``created_at``, ``created_by_user``, ``modified_at``,
    ``modified_by_user``)
  * Actions: ``[ Delete ]`` (if the user has permissions to do so). Open
    ``/relation/<id>/delete/`` to confirm the deletion. See ``Delete`` section.

``[ Add new relation ]`` (would open the **Create** view)

.. note::

   The list of relationships can also be included in the existing view:
   ``/<scope>/<wname>/collection/<source_category>/<source_name>/`` under
   a new HTML header **Relations**. E.g. in the existing
   ``https://debusine.debian.net/debian/base/collection/debian:suite/trixie/``
   after **Artifacts** header.

Create
~~~~~~

(View available if user has permissions to create relations)

* URL structure: ``/<scope>/<wname>/collection/<source_category>/<source_name>/relation/add/``
* URL example: ``/debian/base/collection/debian:suite/trixie/relation/add/``

The view will contain:

  * Title: ``Create relation for <source_name>@<source_category>``

Form with fields:

  * Target: select widget, list collections visible in this workspace (including
    collections from inherited workspaces)
  * Type of relation: ``Fork parent``, ``Requires``, ``Goal``, ``QA Results``

For ``Requires`` relations: the position is not entered manually when creating
it. New relations are appended at the end.

Reorder relations
~~~~~~~~~~~~~~~~~

(View available if user has permissions to edit relations)

  * URL structure: ``/<scope>/<wname>/collection/<source_category>/<source_name>/relation/<relation_type>/edit/``
  * URL example: ``/debian/base/collection/debian:suite/trixie/relation/requires/edit/``

The view will find the ``from`` collection based on the ``scope``, ``workspace``,
``source_category`` and ``source_name``.

The view will display the existing outgoing relations for the given ``from``
collection and ``relation_type`` and allow the user to reorder the ``to``
targets using a UI similar to the inheritance chain widget.

Delete
~~~~~~

(View available if user has permissions to delete the relation, linked from
the ``List of relations`` and/or ``Reorder relations``).

  * URL structure (GET) ``/<scope>/<wname>/relation/<id>/delete/``
  * URL example (GET) ``/debian/base/relation/15/delete/``

  * URL structure (POST) ``/<scope>/<wname>/relation/<id>/delete/``
  * URL example (POST) ``/debian/base/relation/15/delete/``

The GET view will display the relation to be deleted and ask for a final
confirmation, then submit a POST request for the deletion.

If the deleted relation belongs to an ordered relation type (such as
``requires``), the positions of the remaining relations are atomically reindexed
to keep them contiguous.

Permissions
===========

Collections belong to a workspace. Workspace owners can manage the relations
of collections in that workspace.
