The ability to automatically merge non-conflicting changes in two or more branches. Offering a curated merger of conflicts between two decision assets. 

The functionality to group users in hierarchies to enable permission to be allocated at a coarser grain.

The tool is able to support generation of new DRDs based on a DRG. For example, 'nearest 'neighbors' of a specified node, all nodes required to define a specific Information Item or all nodes with a specific hashtag.

The functionality to label model components with security descriptors at a granularity finer than 'entire 'repository', 'entire 'model' or file. The ability to map descriptors to users which administrators can then use to control access to decision models.

The ability to interchange all other functionality supported by the product.

Support for simple Decision Requirement Diagrams which depict Decisions, Input Data and Knowledge Source elements, linked by Information and Authority Requirements. Each element and requirement shape must conform to the DMN specification.

This one covers all boxed expressions as defined by the DMN standard but literal expressions and decision tables, which have their own nodes in the dimension diagram.

The ability to define a library of domain specific functions that are used in a decision model but are standard (to the industry) and should be defined centrally. For example, time value of money, apportionment, Pearson coefficient of correlation, etc.

These are typically not good candidates for decisions because they are mathematical definitions that very seldom change. This is more than being able to call a function from a decision model as it relates to how such functions are defined, found and managed.

This one covers all boxed expressions as defined by the DMN standard but literal expressions and decision tables, which have their own nodes in the dimension diagram.

The ability to define decision logic with execution language that can use structured data (with no side effect), as defined by the item definitions of the decision model, and perform calculations and transformations on it.

The ability to define a vocabulary of single business words and phrases and label them in context. That is, highlight the fact that a sequence of words (e.g., ""cross-funded hedge transaction"") is linked to a dictionary definition.

Part of this is the recognition and demarcation of the use of these terms in the names of DMN nodes, in metadata, in documentation or in logic definitions. Another part is the ability to maintain a dictionary of these terms, remind users of their meaning and support consistent usage.

The ability to support a specified level of access to one node of a decision model and all subordinate nodes related to it.

As above, but supporting the comparison of two or more decision 'models' performance.

The ability to compare two or more decision 'models' performance against the same data set. These might be champion/challenger or A/B evaluation candidates.

The product is able to support multiple concurrent strands of development, for example using branching. Branches can be merged manually.

The extent to which of a decision. This is expressed in two sub-Dimensions:

• Support for visual decision logic. For example: the expression of decision tables, trees, boxed contexts, boxed relations, boxed iterations and other visual representations of logic. Unlike many of the other Dimensions, the progression of these features is less of a linear development and involves more alternative paths.
• Support for textual decision logic. For example: simple literal expressions; nested, structured literal expressions; iterative expressions with contexts, qualifications and other declarative representations of textual logic.

It also requires support for information item data typing: simple types (integer, number, text), support for enumerations, support for collections and support for structured data types.

KPIs are visualized in a dashboard and displayed across a time window or in real-time. Simple filtering (by decision and other qualities) can be performed.

The ability to support DMN decision tables as defined by the standard, which allows decision tables that are not executable as well as ones that are. Decision tables can have cells containing free form text, structured logic or a path language.

The tool is able to show, visually, how two (or more) revisions of the same decision logic are different. The depiction of differences must be contextual (a mere side by side comparison of version A and B is not sufficient). The logic being compared may be in two different branches if branching is supported.

Harvesting decisions (from code) and Mining (from data). Harvesting involves translation of logic from an existing source (e.g., a programming or natural language), whereas mining involves creating logic from data (e.g., using RIPPER to create decision trees from data).

The ability to represent rules in a tree-like fashion, where each internal node is a test on an input, each branch is the outcome of the test, and each leaf is the resulting value to be assigned to the output.

Decision Table Data Entry: This is the facility to check each decision table data entry, not the obligation to use it. The check ensures that each cell of a decision table has an appropriate value that is consistent with its structure and the type of information item expected.

Logic verification: The tool can detect or repair rule problems in the decision table, such as incompleteness, inconsistency, unreachable rules.

Logic simplification: The tool can check or suggest rule improvements to the decision table, such as contraction (merging), reordering, hit indicator transformation, normalization, splitting, refactoring and so on.

The support for automated deployment of decision services to a target architecture from which it can be executed on demand.

Ensure that the DRD and decision logic have consistent connection semantics.

Missing dependencies: The product can inform users if their DRD is inconsistent with the needs of their logic (e.g., the logic definition has an unsatisfied dependency).

Unused dependencies:  The product can prevent dangerous practices such as 'push-'through' dependencies (i.e., information items passed into a decision, A, with the sole aim of passing them to decisions which have an information dependency on A rather than using them itself).

Ensure the DRD is independently sound. Correct use of DMN shapes.

The tool is able to show, visually, how two (or more) revisions of the same DRD are different. The depiction of differences must be contextual (a mere side by side comparison of version A and B is not sufficient). The DRDs may be in different branches if branching is supported.

The ability to interchange the core DMN elements: Decision, Input Data and Knowledge Source. In addition, the ability to interchange other supported nodes, for example BKMs. The metadata includes owner, questions and allowed answers and KPIs (if supported).

The ability to visually wrap one or more nodes in a decision service element that reflects a 'service's public API, the logic it encapsulates (publicly and privately), its internal and external data requirements and the external requirements it have.

The ability to deny access explicitly or implicitly (because a user is not named in and ACL) to named users.

The relative placement of all nodes in the diagram. In addition, this covers additional styling (e.g., color, font).

The ability to interchange information included using the DMN standard extension mechanism.

The ability to use an execution language in a boxed context.

Simple log of all sub-decisions and data leading to each outcome.

The ability to fabricate an explanation for an outcome as an additional output of the decision model itself. This can be done using the decision model to produce extra output (i.e., deliberately collecting additional information to substantiate an outcome). A common example is including the annotations of executed rows in a decision table as a step-by-step explanation. Another technique is to animate the execution of a decision (with movements and color), showing step-by-step how an outcome is derived from inputs and interim results.

Alternatively, tools can use generative networks to create natural language explanations[4] or graphing tools to create visualizations of why a specific outcome was reached in a specific decision instance. Tools could also visualize the border between one outcome and another given the inputs. Typically, this is achieved by graphically representing the relationship between inputs and outcomes.

The ability to define decision logic with free form (non-executable) natural language or pseudo-language or examples.

The ability to associate decisions with business value metrics (e.g., profitability, reduction in client churn, security, failure rate). At this level the KPIs are unregulated and exist for the sole purpose of documentation of business value.

The ability to define decision logic with free form (non-executable) natural language or pseudo-language or examples.

Support for groups (dotted rectangles that encompass one or more shapes) is provided. In addition, the ability to add textual annotations both free-standing or associated with specific groups or elements.

The ability to assist in the merging of multiple development branches of the same decision model into a consolidated branch. Offering a curated merger of two decision assets. 

The option to import and track the origin of and manage named elements.

The ability to depict Business Knowledge Models (BKMs), Knowledge Requirements and the invocation of a BKM in a DRD.

The ability to visually wrap one or more nodes in a decision service element that reflects a 'service's public API, the logic it encapsulates (publicly and privately), its internal and external data requirements and the external requirements it have.

The support for generating and sharing comments on any part of a decision model that is visible to all involved modelers. This feature, akin to social media, might involve integration with a chat platform or email to support contextual discussion of issues and tasks.

The product can be integrated into a broader release management lifecycle, and this may include DevOps support, integration with one or more of the following revision control systems or equivalent: Git, Perforce, ClearCase, SVN etc.

The functionality to link Knowledge Sources to URLs and seamlessly move between decision model and supporting document or entity reference (e.g., email, corporate directory). Users should be able to understand the collection of Knowledge Sources on which their model depends.

The ability to report that a monitored KPI has exceeded (or fallen below) some specified threshold. Some action could also be taken (notification by means of SMS or email, automated behavior, etc.) in reaction to such alarms.

Functionality to interchange a that is linked to decision tables or structured logic and to maintain those links. For instance, a decision table might have a column that is associated with an information requirement to a sub-decision. A complete information model may not be present.

The product is able to support a single, multiuser branch of revisions with commented milestones in the evolution of a decision model.

Machine Learning is a heavily overloaded term, often used for marketing purposes. We will define Machine Learning (ML) here to avoid uncertainty and misunderstanding in this context. A strict subset of artificial intelligence, ML is a statistical approach in which a learner performing a task T, with a performance metric of P, can automatically increase P over time with experience (i.e., past instances of the task, optionally with a correct answer). No explicit programming of the learner is involved; the learner is conditioned by examples only. Common examples of ML are learners that classify email (as spam or ham) or predict individuals' creditworthiness.

Although not defined by the DMN standard, we feel that machine learning (ML) decisions engage in relationships that ordinary decisions do not. For example, an ML decision would require training data, knowledge about the historical statistical distribution of inputs and the ability to learn given experience. Therefore, they merit their own node in the progression graph.

In addition, consider that ML decisions have properties that other decision do not. They have a distinct development and deployment lifecycle. ML decisions using techniques like reinforcement learning, Markov models, ARIMA and Kalman filters have the ability to learn, incrementally from their experience, while they are deployed – they are learning decisions. Unlike , . Hitherto, decision modeling has assumed that decisions are repeatable and this is often a desirable property. Adaptive decisions are not repeatable by design and need to be distinguished, even if only by a subtle variation on the rectangle shape. This important difference may require some visual representation and supporting metadata.

This includes simple KPI monitoring (latency, throughout, profit) and persistence of a KPI value timeseries. It also includes reporting of KPI failure. KPIs therefore must be both measurable and executable. Those KPIs pertaining to ML decisions can include input distribution and covariate drift (to allow a warning to be provided should these KPIs be violated). They can be periodic or real-time. KPI levels and success can be displayed in simple terms. KPI monitoring does not necessarily require decisions to be executable, but the means of measuring the indicator must be specified which is easier in a system supporting implementation.

The ability to search for all the diagrams using a specific node by node name. Both interactive search and a list of search results (a report) is covered by this Dimension.

Tools that support extensions to the DMN standard may also support specific extensions proposed by DecisionAutomation.Org. Such extensions would define a standard way to use the DMN extension mechanism to share a particular kind of information. Support would require a tool to recognize the extension, present the interchanged information as defined by the .Org, and not to use the DMN extension mechanism to interchange the same information in a different way.

The ability to assign model view and editing permissions to a group of users.

The ability to reflect the (recorded) behavior of decision services in production by reference to the original decision model. For example, the ability to depict heatmaps to reveal which decision outcomes predominate, which sub-decision outcomes commonly result in specific decision outcomes etc.

The ability to distinguish between the right to view a model as opposed to being able to edit it.

Facility that tools provide that make models easier to understand e.g. filter out all the rules that lead to one outcome.

• [1] https://www.statistik.uni-dortmund.de/useR-2008/slides/Strobl+Zeileis.pdf
• [2] https://arxiv.org/abs/1602.04938
• [3] https://arxiv.org/abs/2202.05594
• [4] https://www.arxiv-vanity.com/papers/1811.00196/

The ability to retrieve all nodes with metadata that contains a specific text, term or hashtag. For example, show all the nodes that are owned by a specific entity or that have a term in their description.

The ability to search advanced or user-defined metadata for specific terms and hashtags. For example, to find all decisions that contribute to the profitability KPI.

The ability to simulate high volumes of decision instances across the complete data space and yield aggregate statistics about outcomes and performance.

KPIs are defined using a (extendable) set of quantities (e.g., MTBF, client retention, profit) with a strongly typed goal (e.g., percentages, amounts of money, failure rate) and time frame. In other words, they are valid SMART (Specific, Measurable, Achievable, Relevant, Time-bound) goals. These are still for documentation purposes although external tools could utilize this metadata.

Logic expressed in a structured fashion and which should be easy to understand and verifiable by means of an automated process, although not necessarily executable.

For example: find all decisions and decision table rows impacted by 'basket price.discount' for a specific data instance (so some knowledge of the data dependencies on logic calculations would be required).

The ability to represent (using color, icons or other decorations that 'don't interfere with the standard shapes) specific types of Decisions, Knowledge Sources or Input Data defined by the vendor or user. No particular semantics be enforced for styling, in particular when colors are used. For example: tools may add decision icons to depict whether the decision has a logic representation and what type it is (e.g., decision table, path expression); Knowledge Sources might reflect the type of authority (e.g., document, entity, group) and Data Inputs the type of data (e.g., scalar, structure, collection).

In addition, users may use their own icons or colors to denote additional custom semantics.

The entire DRG and individual nodes within it can be documented (using highly-structured, rich text) in a manner that supports the definition of business value, responsibilities, integration and key use scenarios, and optionally allow the generation of a document taking into account the decision model and that rich, structured text.

Each node in a DRG can have relevant metadata attached to it, both the metadata defined by the standard (e.g., Business Owner, Questions and Answers) and custom metadata defined by users. This metadata is accessible from DRDs referencing the node.

Find all decisions with information, authority or knowledge requirements on a specific node.

The ability to define sets of data that accompany decision models. Such data is either: bulk generated examples with inputs only, bulk generated input data examples with outputs to compare against or curated input data examples with expected (correct) outputs.

Curated examples are used to assess the correctness of the model. Specifically, they are designed to show that a model is both correct (in the face of typical input) and robust (given inputs out of their conventional distributions).

Bulk generated examples produce outputs by execution, these outputs can then be compared to the outputs generated by subsequent versions of the model. The goal of this comparison is Impact assessment: to understand the impact of a change. However, unlike curated examples, there is no independent assurance that the output is correct. Naturally, the outputs of bulk generated examples can be reviewed and thereby become curated. It is important to note that correctness testing and impact assessment are distinct uses of test data.

In either case, the datasets allow a degree of confidence that the decision model is working, and in the case of curated test cases, provide a test scorecard indicating which tests executed successfully or matched their expected results.

Logic expressed in a structured fashion and which should be easy to understand and verifiable by means of an automated process, although not necessarily executable.

The functionality to group users in hierarchies to enable permission to be allocated at a coarser grain.

The functionality to record all change actions of each user to provide a full retrospective account of the changes to the model over time.

The product offers facilities to check or improve model soundness.