[QT-696] Add enos scenario outline
#128
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add metadata support to Enos scenarios to allow us to quickly read an
outline of a scenario and determine what quality characteristics are
verified.
We introduce new DSL schema:
- `quality`: A new top-level object that describes a quality
characteristic.
- `quality.description`: An required string attribute in the `quality`
block that describes the quality characteristic in detail.
- `scenario.description`: An optional string attribute that describes
the scenario.
- `scenario.step.description`: An optional string attribute that
describes the scenario step.
- `scenario.step.verifies`: An optional `quality` attribute. Can be
either a singular `quality` or a list of `quality`'s.
Using these new blocks and attributes a scenario author can now more
fully describe the purpose of the scenario, each step, and which quality
characteristics are verified by it.
To see the generated outline you can use `enos scenario outline
<optional-filter>` to generate a scenario outline.
The `scenario outline` sub-command supports a new output formatter:
`html`. Use the HTML formatter and your browser for a cleaner overview
if so desired, e.g.:
```
enos scenario outline --format html > outline.html
open outline.html
```
* Fix deprecations in `.golangci.yml`
* Fix warnings output by `golangci-lint`
* Silence `buf` lint error that is irrelevant to us
* Support decoding `quality` blocks
* Support decoding `scenario.description`
* Support decoding `scenario.step.description`
* Support decoding `scenario.verifies`
* Add `OutlineScenario()` RPC to the enos service.
* Add acceptance test for `enos scenario outline`
* Restructure matrix decoder into its own type. This allows us to keep a
copy of the original matrix around to use in the outline
* Add new decoder target for scenario outlines
* Add `ShowOutline()` to `basic` UI
* Add `ShowOutline()` to `machine` UI
* Create a shim HTML UI
* Add `ShowOutline()` to `html` UI
* Bump version to 0.0.28
* Add missing documenation for `globals` DSL to `README.md`
* Add missing documenation for `sample` DSL to `README.md`
* Add documenation for `quality` DSL to `README.md`
* Add missing documenation for `enos scenario sample` command to
`README.md`
* Add documenation for `enos scenario outline` command to `README.md`
Signed-off-by: Ryan Cragun <[email protected]>
ryancragun
added
the
changelog/feat
Comment on lines
220
to
234
| eMatrix := NewMatrix() | ||
| eAttrs, moreDiags := mBlock.Body.JustAttributes() | ||
| diags = diags.Extend(moreDiags) | ||
| if moreDiags != nil && moreDiags.HasErrors() { | ||
| continue | ||
| } | ||
|
|
||
| for _, attr := range md.sortAttributes(eAttrs) { | ||
| vec, moreDiags := md.decodeMatrixAttribute(ctx, mBlock, attr) | ||
| diags = diags.Extend(moreDiags) | ||
| if moreDiags != nil && moreDiags.HasErrors() { | ||
| continue | ||
| } | ||
| eMatrix.AddVector(vec) | ||
| } |
There was a problem hiding this comment.
Nit: this code appears to be verbatim of the include case. Wonder if this could be shared logic.
There was a problem hiding this comment.
Refactored all matrix attribute decoding into a function which we now use at both top level keys and in include and exclude blocks.
| return v.ShowError(err) | ||
| } | ||
|
|
||
| v.basic.UI().Output(buf.String()) |
There was a problem hiding this comment.
I doubt this will happen but UI() can return a nil pointer and this could panic.
There was a problem hiding this comment.
We have nil guards on the access so it should be safe. I added a test case to check anyway.
Signed-off-by: Ryan Cragun <[email protected]>
Signed-off-by: Ryan Cragun <[email protected]>
Signed-off-by: Ryan Cragun <[email protected]>
Signed-off-by: Ryan Cragun <[email protected]>
jasonodonnell
approved these changes
Apr 9, 2024
Signed-off-by: Ryan Cragun <[email protected]>
Signed-off-by: Ryan Cragun <[email protected]>
ryancragun
requested review from
a team,
marianoasselborn and
jeanneryan
and removed request for
a team
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add metadata support to Enos scenarios to allow us to quickly read an outline of a scenario and determine what quality characteristics are verified.
We introduce new DSL schema:
quality: A new top-level object that describes a quality characteristic.quality.description: An required string attribute in thequalityblock that describes the quality characteristic in detail.scenario.description: An optional string attribute that describes the scenario.scenario.step.description: An optional string attribute that describes the scenario step.scenario.step.verifies: An optionalqualityattribute. Can be either a singularqualityor a list ofquality's.Using these new blocks and attributes a scenario author can now more fully describe the purpose of the scenario, each step, and which quality characteristics are verified by it.
To see the generated outline you can use
enos scenario outline <optional-filter>to generate a scenario outline.The
scenario outlinesub-command supports a new output formatter:html. Use the HTML formatter and your browser for a cleaner overview if so desired, e.g.:.golangci.ymlgolangci-lintbuflint error that is irrelevant to usqualityblocksscenario.descriptionscenario.step.descriptionscenario.verifiesOutlineScenario()RPC to the enos service.enos scenario outlineShowOutline()tobasicUIShowOutline()tomachineUIShowOutline()tohtmlUIglobalsDSL toREADME.mdsampleDSL toREADME.mdqualityDSL toREADME.mdenos scenario samplecommand toREADME.mdenos scenario outlinecommand toREADME.mdChecklist