How to use the CI-Dashboard#

In KnowWE, continuous integration facilitates the automated testing and building of knowledge bases. Since, we provide a distributed knowledge base development, it is important to install simple mechanisms to automatically control the quality of the knowledge base after every change or on demand.

Installation#

The continuous integration plugin is installed by

%%CIDashboard
  @name: <dashboard name>
  @trigger: <onSave/onDemand> <article for onSave trigger>
  @test: <testname> <test object> <parameters (if required)...>
%

Please use the annotations of the plugin as follows:

Annotation Mandatory Description
name yes Specify the name of the wiki article, where the knowledge base is defined
trigger yes Controls the time, when the integration is executed: onSave executes a new integration after any save action in the wiki (recommended) and onDemand only executes the integration when the user explicitly starts it in the tool menu of the continuous integration plugin (only recommended for large knowledge bases and slower computers).
test optional Specify the name of the test and its parameters. To define more than one test, you may use the test annotation multiple times.

Available Tests#

In this installation the following tests are available. However, new tests can be added by new plugins.

Name of TestTest ObjectDescription
ArticleContainsArticleSynopsis: @test ArticleContains «Article» «SearchString»

Checks, whether the articles contain the text defined by the search string.

Parameter Details:

  • «SearchString»:
    Specifies the string or pattern (regex) that will be searched. Any found occurrence within the test object will be considered as test failure.
ArticleHasErrorsArticleSynopsis: @test ArticleHasErrors «Article»

Checks, that the specified articles reports no compile errors.

ArticleHasWarningsArticleSynopsis: @test ArticleHasWarnings «Article»

Checks, that the specified articles reports no compile warnings.

DiaFlux:AmbiguousEdgeKnowledgeBaseSynopsis: @test DiaFlux:AmbiguousEdge «KnowledgeBase»

If multiple outgoing edges of a node use the same choice, the edges of this node become ambiguous. In this case the flow may divert into two parallel paths which is often not desired.

DiaFlux:CombinedFlowRuleAbstractionKnowledgeBaseSynopsis: @test DiaFlux:CombinedFlowRuleAbstraction «KnowledgeBase»

Checks whether there is an abstraction question, that is derived by an abstraction rule as well as by a DiaFLux action node - this may cause TMS conflicts.

DiaFlux:CombinedFlowRuleSolutionKnowledgeBaseSynopsis: @test DiaFlux:CombinedFlowRuleSolution «KnowledgeBase»

Checks, whether there is a solution for which states are assigned by a DiaFlux node AND a scoring rule as this might cause truth-maintenance problems.

DiaFlux:ConflictingAssignmentKnowledgeBaseSynopsis: @test DiaFlux:ConflictingAssignment «KnowledgeBase» [«timeout»]

Tries to find a path, where one question is first conditioned by a guard of an edge and later a new value is assigned to this question. The assignment of the new value will retract the path until the previous condition and consequently will yield a infinite reasoning cycle. For this reason, before setting a new value to a question, that was previously conditioned, a snapshot node has to be placed.

Parameter Details:

  • «timeout» (optional):
    timeout in milliseconds when the search process for conflicting assignments will be terminated.

This test also allows to specify items to be ignored:

  • «ignore objects» (optional):
    Knowledge objects with a name matching this regular expression are excluded from the conflicting assigenment search.
DiaFlux:CyclicFlowKnowledgeBaseSynopsis: @test DiaFlux:CyclicFlow «KnowledgeBase» [«timeout»]

This cyclic flowchart test examines all DiaFlux flowcharts of a given knowledge base for any cyclic dependency. A cyclic dependency is available if there is any node form that you can follow the diagnostic path towards the node itself. If there is any snapshot in between, or any node that repeatedly asks a specified question, the path is not assumed to be cyclic, because after re-answering a question you may follow a different path. Thus you are not 'captured' in the cycle when performing the interview.

Parameter Details:

  • «timeout» (optional):
    timeout in milliseconds when the search process for cycles will be terminated.

This test also allows to specify items to be ignored:

  • «flowchart»:
    The flowchart a node or edge that shall be ignored in. You must specify the id or name of the flowchart.
  • «node-or-edge»:
    Id of the node or edge that shall be ignored in the flowcharts.
DiaFlux:DuplicatePathsKnowledgeBaseSynopsis: @test DiaFlux:DuplicatePaths «KnowledgeBase» [«min-path-size»]

This tests searches for duplicate sub-paths within the flowcharts of a knowledge base. A duplicate sub-path is a set of nodes and interconnection edges between the nodes of a flowchart, that is identically found at the same or an other flowchart.

Parameter Details:

  • «min-path-size» (optional):
    Defines the number of nodes a duplicate sub-path must at least consist of, to be considered as a duplicate path. Default value is '2'
DiaFlux:InformationGainKnowledgeBaseSynopsis: @test DiaFlux:InformationGain «KnowledgeBase» «min-gain-average» «min-gain-each» [«attach»] [«timeout»]

The information gain test checks every flowchart node that asks a question for the information gain achieved by the particular question. The information gain for each question node is measured by all solutions that can be reached by following the flowchart(s) from the question node and how the number of these solutions is reduced by answering this particular question. Each information gain is compared against the maximum information gain of the node, calculated depending on the number of outgoing edges. You can compare both the average information gain and the particular information gain of each asking node against the specified limits.

Parameter Details:

  • «min-gain-average»:
    specifies the minimum percentage of the possible information gain to be reached by all decisive questions in average, e.g. 0.1 or 10%.
  • «min-gain-each»:
    specifies the minimum percentage of the possible information gain to be reached by each decisive question, e.g. 0.1 or 10%
  • «attach» (optional):
    specifies if the detailed result table of the evaluation shall be attached or not. Default value is 'false'.
  • «timeout» (optional):
    timeout in milliseconds when the calculation of the average path length will be terminated.
DiaFlux:InterviewLengthKnowledgeBaseSynopsis: @test DiaFlux:InterviewLength «KnowledgeBase» «max-length-average» «max-length-each» [«attach»] [«timeout»]

The interview length test check the number of questions asked during the interview until a solution is reached. You may specify both, the average interview length that should not been exceeded in average and a maximum interview length that should also not been exceeded by the longest interview path.

Parameter Details:

  • «max-length-average»:
    specifies the maximum interview length that should not be exceeded in average.
  • «max-length-each»:
    specifies the maximum interview length that should not be exceeded for every particular solution.
  • «attach» (optional):
    specifies if the detailed result table of the evaluation shall be attached or not. Default value is 'false'.
  • «timeout» (optional):
    timeout in milliseconds when the calculation of the average path length will be terminated.
DiaFlux:MissingEdgeKnowledgeBaseSynopsis: @test DiaFlux:MissingEdge «KnowledgeBase»

Searches all nodes in the DiaFlux models of a knowledge base that with outgoing edges does not cover all possible values. In this case the flow may stop unexpectedly in between.

DiaFlux:NoAutostartKnowledgeBaseSynopsis: @test DiaFlux:NoAutostart «KnowledgeBase»

Tests whether exists an autostart flow in the DiaFlux model.

DiaFlux:StandardQuestionDerivedByAbstractionKnowledgeBaseSynopsis: @test DiaFlux:StandardQuestionDerivedByAbstraction «KnowledgeBase»

Checks whether there exists a standard question (not an abstraction question), that is derived by an abstraction rule or a DiaFlux action node.

DiaFlux:UnusedFlowKnowledgeBaseSynopsis: @test DiaFlux:UnusedFlow «KnowledgeBase»

This test checks whether the knowledge base contains DiaFlux models, that are neither Autostart models nor are called by other DiaFlux models

This test also allows to specify items to be ignored:

  • «flows»:
    A regular expression naming those DiaFlux models to be excluded from the tests.
DiaFlux:UnusedNodeKnowledgeBaseSynopsis: @test DiaFlux:UnusedNode «KnowledgeBase»

Looks for non-connected start or exit nodes in DiaFlux models.

EmptyQuestionnaireKnowledgeBaseSynopsis: @test EmptyQuestionnaire «KnowledgeBase»

Tests whether the knowledge base has questionnaires that do not contain any questions or other questionnaires.

This test also allows to specify items to be ignored:

  • «questionnaire»:
    A regular expression naming those d3web qcontainers to be excluded from the tests.
FunctionalPropertyObjectSynopsis: @test FunctionalProperty «Object»

Checks for all existing functional property whether they are indeed used as functional, i.e., for each subject node only one object exists.

IncompleteChoiceQuestionKnowledgeBaseSynopsis: @test IncompleteChoiceQuestion «KnowledgeBase»

Looks for choice questions that have no choice alternatives defined yet.

SingleTermDefinitionArticleSynopsis: @test SingleTermDefinition «Article»

This test checks, if each term is defined only once.

This test also allows to specify items to be ignored:

  • «terms» (optional):
    Term names to ignore for this test.
SparqlResultSizeObjectSynopsis: @test SparqlResultSize «Object» «comparator» «expected size»

Checks the size (amount of rows) of a given sparql query result against an expected value.

Parameter Details:

  • «comparator»:
    how to compare the result size against the expected size
  • «expected size»:
    expected size to compare with
SparqlResultTableObjectSynopsis: @test SparqlResultTable «Object» [«comparator»]

Test a expected sparql result set against the result set of the actually executed sparql query.

Parameter Details:

  • «comparator» (optional):
    how to compare the result data against the expected table data: equal/atleast
TermnameConventionArticleSynopsis: @test TermnameConvention «Article» «Naming convention pattern»

Tests whether the objects names used on this article comply to the specified naming convention pattern.

Parameter Details:

  • «Naming convention pattern»:
    A regular expression for the desired naming convention. All terms are checked against this expression.
TestCaseTestCaseSynopsis: @test TestCase «TestCase» «KnowledgeBase» [«ValueOutOfRange»]

This test executes test cases. It compares the expected findings defined in the test cases with the findings actually derived by the knowledge base.

Parameter Details:

  • «KnowledgeBase»:
    Specifies the knowledge base with which the test case is to be tested.
  • «ValueOutOfRange» (optional):
    Specifies whether values out of the defined ranges should be skipped or not. By default they are not ignored.
URIPatternTestObjectSynopsis: @test URIPatternTest «Object»

Checks for all Terms (URIs) whether they follow best practice URL patterns.

This test also allows to specify items to be ignored:

  • «Ignore-Pattern»:
    Specifies a regular expression of all terms to be ignored. Please note that the term is similar to the abbreviated uri usedin turtle markups, but uses '#' as a separator.

Example#

%%CIDashboard
  @name: MyArticle CI Test
  @trigger: onSave MyArticle
  @test: EmptyQuestionnaireTest MyArticle
  @test: IsolatedInterfaceNodeInDiaFluxModel MyArticle
  @test: NoAutostartFlowCITest MyArticle
  @test: StandardQuestionDerivedByAbstraction MyArticle
  @test: CombinedDiaFluxRuleAbstraction MyArticle
  @test: StandardQuestionDerivedByAbstraction MyArticle
  @test: IncompleteChoiceQuestion MyArticle
%

Looks like:

Screenshot of a CIDashboard

Related Doc Pages#

Add new attachment

Only authorized users are allowed to upload new attachments.

List of attachments

Kind Attachment Name Size Version Date Modified Author Change note
png
CIDashboard-screenshot.png 58.3 kB 1 26-Oct-2012 10:15 Jochen Reutelshöfer
This page (revision-2) was last changed on 18-Dec-2012 00:46 by Albrecht Striffler