SOUP Recipe
Table of contents:
Introduction
Recipes are a fundamental concept within the Bosch IoT Rollouts System Software Update (SOUP) extension. They serve as system update definitions and encompass crucial information, including:
Identification of the targeted systems for the update
Specification of the software to be installed on each module
Definition of the desired target state of the system after the update has been applied
Optional pre- and post-system update scripts
Inclusion of user-related information within the recipe metadata
Meta information such as the publishing date or tags associated with the recipe
Recipes play a central role in orchestrating and managing system updates, providing a comprehensive framework for ensuring efficient and effective software updates within the Bosch IoT Rollouts ecosystem.
Recipe data model
A recipe is a system update definition, that specifies a set of matching modules, references distribution sets, and describes the desired system state after the recipe has been applied. Besides basic properties (name, version, and description of the recipe), the recipe has
Audit information detailing creation, modification, publication, and approval
Lifecycle status ( refer to Recipe lifecycle)
Validity status ( refer to Recipe validation)
Module update definitions for identifying matching system ( refer to Module update definition), and references to software for module updates, if applicable
System distribution sets for referencing non-module-related artifacts (e.g., system release notes, pre- or post-install scripts) ( refer to System distribution set)
Metadata for providing additional custom information (primarily used during transformation to the recipe file)
Tags for efficient filtering within the backend or on the Install API
Module update definition
A module update definition consists of a three elements:
The module query (a.k.a. Module match) identifies a set of modules (based on properties from the controller attributes).
The count specifies the expected number of matching modules within a system.
The optional distribution set specifies the software that should be applied to the matching modules. If no distribution set is specified, the matching modules need to be present in the system but will not be updated.
Module match query syntax
The module match queries are similar to target filter queries, however, they only support a subset as given in the table below:
|
Operator |
Description |
Example |
|
== |
equals |
attributes.module-id==4711 |
|
!= |
not equals |
attributes.iso-code!=uk |
|
=lt= or < |
less than |
attributes.hwVersion=lt=3.7.1 |
|
=le= or <= |
less then or equals |
attributes.partNumber=le=1337 |
|
=ge= or >= |
greater than or equals |
attributes.SwRevision=lt=5.1-beta |
|
=gt= or > |
greater than |
attributes.Ver=lt=3.7.1 |
|
=in= |
equals one entry in the list |
attributes.hwVersion=in=(1.3,1.4,2.5) |
|
=out= |
does not equal any entry in the list |
attributes.hwVersion=out=(3.1-beta,5.2-rc) |
|
and or ; |
logical and composition of two terms. AND in uppercase is not supported |
attributes.module-id==4711 and attributes.hwVersion=in=(1.7,1.8,2.5) |
All attributes are strings. However, using string compare for versions does not make sense. Therefore, the following sorting is applied:
Attributes Ver of *Version (e.g. Ver, hwVersion) use natural sort order
All other files use string compare.
Module count query syntax
|
Operator |
Description |
Example |
Explanation |
|
== |
equals |
count==1 |
Mandatory module that must be there exactly once in the system. |
|
=lt= or < |
less than |
count=lt=2 |
Optional module that can be maximum once in the system. |
|
=le= or <= |
less then or equals |
count=le=2 |
Optional module that can be maximum twice in the system. |
|
=ge= or >= |
greater than or equals |
count=ge=0 |
Optional module that can be part in any number within the system. |
|
=gt= or > |
greater than |
count=gt=0 |
Mandatory module, that must be at least once in the system. |
System distribution set
A system distribution set specifies distribution sets that are required on system level (e.g. system release notes, install scripts).
Recipe match algorithm
The recipe match algorithm describes how a matching recipe for a given system is identified. For a successful match, a recipe has a module update definition for every module within the system, satisfying both the match query and count requirements . In cases where multiple recipes could potentially match a system, the algorithm selects the most recent one based on its publish date. The algorithm is triggered automatically for
all systems, if a recipe changed its status (i.e., on promote, demote, approve, revoke, deactivate),
on a single system, if a system is updated (e.g. a new system report has been received).
After the recipe match job executed the algorithm, all systems have a matching recipe and/or matching release candidate recipe depending on the output of the job.
Recipe coverage
When a recipe is activated, it is crucial for the update coordinator to ascertain the number of systems that will be impacted by the recipe. This information is valuable in assessing both the extent of coverage and the potential consequences in case of any issues. After the execution of the recipe match, the coverage of the recipe is presented on a relative scale ranging from very low to very high, depending on the number of matching systems.
|
Icon |
Name |
Description |
|
|
Very high |
The recipe matches more than 80% of the systems. |
|
|
High |
The recipe matches between 61% and 80% of the systems. |
|
|
Medium |
The recipe matches between 41% and 60% of the systems. |
|
|
Low |
The recipe matches between 21% and 40% of the systems. |
|
|
Very low |
The recipe matches less than 20% of the systems. |
|
|
No |
No system matches the given recipe. |
|
|
In progress |
The coverage calculation is in progress. |
Recipe lifecycle
The lifecycle of a recipe can be controlled via the Recipe UI or Management API. Lifecycle management enables System responsibles to
release a recipe for testing,
release and approve a recipe for productive system updates,
take a recipe offline which is suspected to cause issues in the field, or
retire a recipe which is no longer needed (because of newer recipe versions).
Defined recipe states
|
Icon |
State |
Description |
Exposed on Install API |
Modifiable |
Deletable |
|
|
DRAFT |
Initial state right after creating (or uploading) the recipe. |
No |
Yes |
Yes |
|
|
WAITING_FOR_SIGNATURE |
Recipe file is created and the signature needs to be uploaded. Recipe must be valid to switch into that state |
No |
No* |
No |
|
|
RELEASE_CANDIDATE |
Recipe is ready for verification testing. |
Yes - scope testing |
No* |
No |
|
|
WAITING_FOR_APPROVAL |
Recipe has passed all verification tests and can be released once it has been approved by an approver. |
Yes - scope testing |
No* |
No |
|
|
RELEASED |
Recipe is released and can be applied to productive systems. |
Yes - scope production |
No* |
No |
|
|
INACTIVE |
A released recipe which has been deactivated may still be used for system updates. For example, a newer version of the recipe is available. |
Yes - scope production |
No* |
Yes |
|
|
REVOKED |
A released recipe which has been revoked must not be used for system updates. For example, the recipe needs to be verified due to a potential error/bug. |
Yes - scope production |
No* |
Yes |
*) Recipe content is read-only, but tags can still be modified.
Recipe state transitions
The following picture illustrates the lifecycle of a recipe. The state transition actions are typically triggered by a system responsible or update coordinator who is authorized to do so.
Explanatory notes:
When a recipe is promoted to RELEASE_CANDIDATE, it becomes accessible via the Install API. Draft recipes, on the other hand, are only accessible through the Management API.
Uploading a signature during the WAITING_FOR_SIGNATURE state automatically moves the recipe to the RELEASE_CANDIDATE state.
The release action is used to initiate the release of the recipe, triggering a request for final approval.
The approval action, which is used to push a recipe into RELEASED state, requires a special permission.
Once a recipe is in the REVOKED or INACTIVE state, it must be re-approved to be put back into the RELEASED state.
Deletion of recipes, applicable only for recipes in the DRAFT, REVOKED, or INACTIVE state, requires special permission.
Recipe validation
Recipe validation is a crucial step in ensuring the integrity and reliability of recipes before they are exposed to the systems through the Install API. Its primary purpose is to verify that the management representation of a recipe can be successfully transformed into a valid install representation, also known as the recipe file. By undergoing the validation process, recipes are thoroughly examined to identify any potential faults or errors. This validation is a prerequisite for promoting a recipe from the DRAFT stage, as only valid recipes can proceed further in the lifecycle. Through recipe validation, SOUP guarantees that only syntactically correct and complete recipes are made available for system updates, enhancing the overall stability and reliability of the update process.
Available validations
Following is a list of available recipe validations and whether they are generic or can be configured for each tenant.
|
Validation |
Default |
Description |
|
Distribution set |
Yes |
The distribution set validation checks, if the referenced distribution sets exist and that they are complete. |
|
Match query |
Yes |
The match query validation checks that the provided module query is syntactically correct (e.g. only supported operators are used). |
|
Metadata |
No |
Metadata validators allow to specify
|
|
Total file size |
No |
The total file size validator allows to configure a maximum total artifact size of a recipe. In addition, a list of software module types can be provided, that shall be excluded from the calculation (e.g. release notes) For example, the gateway can only process updates smaller than 100 Mb. |
Metadata validation rules are configured per subscription using the Management API
Validity states
|
Icon |
State |
Description |
|
|
PENDING |
There is currently no validation result available. It is either a new recipe, or was changed since the last validation. |
|
|
IN_PROGRESS |
The validation is ongoing. |
|
|
VALID |
The recipe has successfully passed the validation. If the recipe is in DRAFT state it can be promoted now. |
|
|
INVALID |
There are validation errors. Please check the validation result. After the errors have been rectified, a re-validation can be triggered. |
Validity state machine
Recipe file
Introduction
The install representation of a recipe, also known as recipe file, is specifically designed to optimize the execution of system updates through a system update handler, typically operating within a gateway. This representation is derived from the management representation and subsequently enhanced with essential information required by the system update handler. While, for example, the management representation of a recipe solely references distribution sets, the recipe file provides comprehensive details about the individual artifacts contained within these distribution sets. This includes information such as byte size, hashes, and download information. The recipe file is generated, when promoting the recipe from DRAFT stage to WAITING_FOR_SIGNATURE.
Default recipe file format
By default, the recipe adheres to a SWupdate-compatible format (sw-description.json), augmented with custom attributes. However, it is important to note that the install representation can be customized to meet specific customer requirements, ensuring flexibility and adaptability. Properties starting with an _ (e.g. _noImage) are ignored by SWupdate.
The recipe file format is configurable per subscription.
|
Key |
Source |
Comment |
|
_recipe.id |
Recipe Id |
|
|
_recipe.schema-version |
Template |
Hard-coded version of the current template. If the template is changed later on, the version should be adapted as well. |
|
_recipe.name |
Recipe name |
|
|
_recipe.description |
Recipe description |
|
|
_recipe.published |
Recipe published dated |
Date is set, when the recipe is promoted from draft to release candidate |
|
software.version |
Recipe version |
|
|
software.scripts |
refer to scripts |
The scripts section contains all artifacts within software modules within a distribution set of type "scripts" referenced as System distribution set Functional System-level scripts that are executed by the SWUpdate application on the gateway (e.g. example.lua). |
|
software.images |
refer to images |
The images section contains all modules of the Module update definitions with a distribution set reference. Functional Modules that are updated in the scope of the system update including their cardinality, and the software that is to be installed on them. |
|
software._noImage |
refer to _noImage |
The _noImage section contains all modules of the Module update definitions without a distribution set reference. Functional Modules are not updated in the scope of the system update including their cardinality. |
|
software._systemArtifacts |
refer to _systemArtifacts |
The _systemArtifacts section contains all artifacts within software modules within distribution sets referenced as System distribution set (incl. scripts). Functional System-level artifacts which are downloaded by the System update-app (e.g. system release notes, scripts or license agreements) |
scripts
The scripts section contains all artifacts within software modules within a distribution set of type "scripts" referenced as System distribution set
|
Key |
Source |
Comment |
|
software.scripts[].id |
Id of file |
Used for retrieval of file via Install API |
|
software.scripts[].type |
Software module type key |
The software module type key has to correspond to the required SWUpdate handler that handles the script |
|
software.scripts[].byteSize |
Size of file |
|
|
software.scripts[].filename |
Name of file |
|
|
software.scripts[].sha256 |
SHA256 of file |
|
images
The images section contains all modules of the Module update definitions with a distribution set reference.
|
Key |
Source |
Comment |
|
software.images[]._module.name |
Module update definition name |
|
|
software.images[]._module.description |
Module update definition description |
|
|
software.images[]._module.match |
Module update definition match query |
"attribute." prefix is removed. Query is then transformed from a RSQL to MongoDB query. |
|
software.images[]._module.matchCount |
Module update definition count query |
|
|
software.images[]._software.softwareId |
Distribution set Id |
|
|
software.images[]._software.name |
Distribution set name |
|
|
software.images[]._software.artifacts |
List of all artifacts with all software modules within this distribution set. |
|
|
software.images[]._software.artifacts[].id |
Id of file |
Used for retrieval of file via Install API |
|
software.images[]._software.artifacts[].type |
Software module type key |
|
|
software.images[]._software.artifacts[].byteSize |
Size of file |
|
|
software.images[]._software.artifacts[].filename |
Name of file |
|
|
software.images[]._software.artifacts[].sha256 |
SHA256 of file |
|
|
software.images[].type |
Template |
Module update definition meta data imageType |
|
software.images[].filename |
Filename of artifact within software module of software module type "firmware" |
Module update definition meta data imageFileName |
|
software.images[].version |
Distribution set version |
|
|
software.images[].sha256 |
SHA256 of file "software.images[].filename" |
|
_noImage
The _noImage section contains all modules of the Module update definitions without a distribution set reference.
|
Key |
Source |
Comment |
|
software._noImage[]._module.name |
Module update definition name |
|
|
software._noImage[]._module.description |
Module update definition description |
|
|
software._noImage[]._module.match |
Module update definition match query |
"attribute." prefix is removed. Query is then transformed from a RSQL to MongoDB query. |
|
software._noImage[]._module.matchCount |
Module update definition count query |
|
_systemArtifacts
The _systemArtifacts section contains all artifacts within software modules within distribution sets referenced as System distribution set (incl. scripts).
|
Key |
Source |
Comment |
|
software._systemArtifacts[].softwareId |
Distribution set Id |
|
|
software._systemArtifacts[].name |
Distribution set name |
|
|
software._systemArtifacts |
List of all artifacts with all software modules within this distribution set |
|
|
software._systemArtifacts[].artifacts[].id |
Id of file |
Used for retrieval of file via Install API |
|
software._systemArtifacts[].artifacts[].type |
Software module type key |
|
|
software._systemArtifacts[].artifacts[].byteSize |
Size of file |
|
|
software._systemArtifacts[].artifacts[].filename |
Name of file |
|
|
software._systemArtifacts[].artifacts[].sha256 |
SHA256 of file |
|
Recipe file signature
Once a recipe is promoted from the DRAFT stage to WAITING_FOR_SIGNATURE, the recipe file becomes available for downloading and signing. To ensure the authenticity and integrity of the recipe file, signatures can be uploaded either through the Recipe UI or Management API. This allows systems or the system update app to verify the validity of the downloaded recipe file. When accessing the recipe file via the Install API, the corresponding signatures can also be downloaded. In scenarios where multiple signatures are available, it is the responsibility of the gateway to select the appropriate signature that it can confirm. This mechanism enhances the security and trustworthiness of the recipe file, ensuring that only authorized and verified updates are executed by the system.
The signature can be created using the Bosch IoT Rollouts Extension - Sign & Encrypt.