Relationship type hiearchy

[lauwers]

During yesterday's TOSCA Community Meeting, @tliron made a comment that while Kubernetes doesn’t formally define relationships, it does have the concept of ownership relationships, which as he mentioned is used primarily for garbage collection: if the owning entity is deleted, then the owned entities should be deleted as well. In UML, this type of relationship is referred to as composition containment where the containment semantics specify that the lifecycle of the contained entities is determined by the lifecycle of the containing entity. This type of relationship is very different from other types of relationships (such as simple dependencies) that don’t have the same lifecycle coupling as containment relationships.

For purposes of Orchestration, making a distinction between containment relationships vs simple dependency relationships is extremely helpful for Day-2 scenarios where orchestrators may need to make decisions about how to propagate events across the service topology graph. For example, if a containing node goes into an error state (or a deleted state), then by definition the contained nodes go into that same state as well. This may not be the case for nodes that have simple dependency relationships to the failed or deleted nodes.

Based on this, I firmly believe that we should reflect this distinction between relationship types in the TOSCA Relationship Type hierarchy. I use this in the Ubicity profiles, which is documented in https://github.com/oasis-open/tosca-community-contributions/blob/master/profiles/com/ubicity/core/1.0/README.md#relationship-types. I’m hoping we can adopt this approach in any community profiles we define.

[tliron]

I generally agree with your suggestion.

The concept here has to do with how orchestrators manage relationships in regards to the source and target nodes.

The simple solution is to add a keyname to the relationship type entity, something like contains: true.

But I think a better idea is to add directives to the relationship type entity, like we have for node templates (but not node types; question - should node types have this, so that the templates would automatically "inherit" them?). In the future we might want to support additional directives, or indeed just let users create custom ones that suit their orchestration implementation.

The particular directive in this case can be "owns-target". But there can also be "owns-source", or "owns-mutual" (which signifies that if either source or target are deleted then the other would also have to be deleted).

[lauwers]

I prefer the keyname approach. In fact, in my implementation I use a non-standard keyname called kind that can have one of three values: ASSOCIATION, DEPENDENCY, or CONTAINMENT. I enforce derivation rules where derived types are not allowed to change the relationship kind of their parent type. This approach focuses on the semantics of the relationship, rather than on providing instructions to the orchestrator, which I think is more portable. In practice, I haven't found a need for other kinds of relationships.

[tliron]

It might be best to not define anything at all in the language.

This could be entirely handled via TOSCA metadata, letting profiles/orchestrators use whatever definitions make sense for their domains.

[lauwers]

I am perfectly fine with that, especially since this is a discussion about TOSCA Community Profiles rather than about possible extensions to the TOSCA language. That said, I feel that the distinction between these different kinds of relationship types is sufficiently important that it should bubble up to the top of any relationship type hierarchy we adopt in community profiles. I would again recommend that we start with something like https://github.com/oasis-open/tosca-community-contributions/blob/master/profiles/com/ubicity/core/1.0/README.md#relationship-types

[tliron]

I would say the idea is very worthy of experimentation. Let's start with metadata and find out how far it takes us.

[pmjordan]

I certainly agree that there are different types of relationship and orchestrator behavior is likely to differ depending on the type in the ways you have described. However I would point out that using metadata to indicate those types would appear to be contrary to the spec as 5.3.1 says "metadata ... MAY be ignored by TOSCA Orchestrators and SHOULD NOT affect runtime behavior"

[tliron]

That "SHOULD NOT" warning is somewhat odd to me. I guess the author is trying to make the point that a property or attribute should be used instead? I guess that could work in this case, too. Especially the value keyname of a property can be used to ensure that derived types don't change the behavior. Also worth noting that metadata is not inherited by derived types.

OK, I'm convinced.

[lauwers]

I interpret that warning to mean that orchestration semantics should only be based on keywords defined in the TOSCA language, not on any metadata. More importantly, just like in any other computer language, orchestration semantics should never be based on values assigned to variables defined using that language. I strongly object to any conventions that based orchestration decisions based on values of properties.

[tliron]

I think you may be using a very restricted meaning of "semantics" here. Properties are general purpose, designed to "configure" the orchestrator's behavior, whatever that may mean. Whether behavior includes "semantics" (such as ownership) or not may depend on how the orchestrator works.

In any case, we can provide all the warnings we want in the spec, but users will use whatever feature helps them solve a problem.

[Roberto-Mameli-Eng]

Hi Chris, Tal, and all

could this be managed by definining new Data Types as below?

data_types:
  RelationshipKind:
    derived_from: string
    validation:
      $valid_values: [containement,dependency,association]
...
  CapabilityKind:
    derived_from: string
    validation:
      $valid_values: [container,feature,partner]

and then defining respectively 3 Relationship types and the corresponding Capability Types (derived from Root) for each of the values above (example below for container/containement):

capability_types:
  Container:
    derived_from: Root
	valid_relationship_types: [Containement]
    properties:
      kind:
        type: CapabilityKind
		required: true
		value: container
...

relationship_types:
  Containement:
    derived_from: Root
	valid_capability_types: [Container]
    properties:
      kind:
        type: RelationshipKind
		required: true
		value: containement
    interfaces:
	  # Operations/Notifications needed for relationship handling

This approach follows the standard and does not enforce customizations on the Orchestrator. Proper handling of relationship behavior between source and target nodes is enforced through operations/notifications (ultimately, through custom artifacts outside of the orchestrator).

Does it make sense? Am I missing something?

If the approach works, definitions as those above may be a starting point for our common profile.

Roberto

[lauwers]

I think there is a difference between TOSCA processing which is based exclusively on the semantics of the TOSCA language and the execution of interface operations which is done using custom artifacts (outside of the orchestrator as you correctly pointed out). TOSCA processing (in my opinion) exclusively manipulates the service representation graph. It does not interface with any external resources. That is the domain of the interface operations: these are primarily responsible for reflecting the state of the service representation graph into the external resources and vice versa.
What you're suggesting is to use interface operations also for changing the state of the service representation graph by having those operations implement the logic for how to propagate events and/or errors across the graph. I see two problems with that:

1. This requires exactly the type of customization that you're trying to avoid: every service designer will have to reimplement this logic for their specific application.
2. This assumes that interface operations have access to the entire service representation graph. This is not assumed in the TOSCA specification, since the membrane between interface operations and nodes and relationships in the graph is through operation inputs and outputs only. I know that some implementations allow interface operations to manipulate the graph directly, but this is technically non-standard behavior.

In practice, I believe there are only a small number of ways in which events and errors should be propagated, and based on my experience it is easy to distinguish between these different ways by making decisions based on relationship kinds. I believe that these decisions can be made in a general-purpose way without taking domain-specific information into account which would then allow general-purpose TOSCA processing behavior without any domain-specific customizations. Of course I could be wrong, and I welcome any examples that show that a general-purpose approach may not be possible.

[tliron]

I half agree. :)

Your point about event propagation, which includes errors, is the crux of the discussion in my view. If we figure out how events work and if and when they propagate through relationships, then there may be room to provide TOSCA keynames to guide those events.

What I prefer not to do at this stage is to define these events and their semantics in the language. "Deletion" might seem simple, but every resource platform understand and manages it differently. Indeed, for some platforms "delete" is merely a user request that may or may not succeed. Even Kubernetes provides hooks as well as other mechanisms to prevent deletion. Errors are indeed a big part of handling, and we have none of that defined in TOSCA right now.

If we can find a way to define events in a generic way, then it would be up to the orchestrator to handle them. It could indeed be via an interface that acts as a "sink" for events.

I am looking forward to working together on an event model for TOSCA 3.0. :)