Kodi Arfer / Research projects / Artiruno

Artiruno documentation



This manual describes Artiruno version 0.4.3.

Artiruno is a Python package for performing verbal decision analysis (VDA) in the manner of ZAPROS (Larichev & Moshkovich, 1995) and UniComBOS (Ashikhmin & Furems, 2005). VDA is a class of decision-support software for multiple-criteria decision-making that doesn't ask the user for explicit preference functions or criterion weights. Instead, the user is asked questions such as "Which of these two items would you prefer?". VDA doesn't always produce a total order for the alternatives, but the weaker assumptions made about what people are capable of accurately judging helps to ensure that results aren't contaminated by arbitrary choices of numbers and functions.

Artiruno is described in more detail and empirically assessed in a paper in the Journal of Multi-Criteria Decision Analysis (Arfer, 2024).


Artiruno has a web interface and a terminal-based interface. It can also be used programmatically. To install the latest release from PyPI, use the command pip3 install artiruno. You'll need Python 3.8 or greater. You can then start an interactive VDA session with python3 -m artiruno FILENAME; say python3 -m artiruno --help to see the available command-line options.

Artiruno input files are formatted in JSON. See the examples directory of the source code for examples, and artiruno.avda() below for a description of the arguments.

Artiruno has an automated test suite that uses pytest. To run it, install the PyPI packages pytest and pytest-asyncio and then use the command pytest. To run it on a web browser with Pyodide, use the script pyodide_testing.py. By default, regardless of platform, particularly slow tests are skipped; say pytest --slow to run all the tests.

The documentation uses Sphinx. To build it, say sphinx-build -b html doc/ doc/_build/.

How it works

Given a set of alternatives and a set of criteria on which the alternatives can be rated, Artiruno can try to find the best single alternative or the best n alternatives, or it can try to rank all the alternatives. See Arfer (2024) for details. Once the goal has been achieved, or Artiruno can't find any more useful questions to ask, Artiruno will return the inferred preferences. In interactive mode, the top-n subset of your preferences (as defined by artiruno.PreorderedSet.extreme()) will be shown if requested, and your preferences will be displayed as a graph with GraphViz if the corresponding Python package is available. (Graphs are not currently implemented for the web interface.)

API reference

Verbal decision analysis

artiruno.vda(criteria, alts=None, asker=None, find_best=None, max_dev=2, allowed_pairs_callback=<function <lambda>>)

Conduct verbal decision analysis.

  • criteria -- An iterable of iterables specifying the levels of each criterion. Levels can be any hashable object, but are typically strings. Within a criterion, we assume that later levels are better.
  • alts -- An iterable of the alternatives; that is, the specific items that can be decided among. Each alternative is represented as an iterable of criterion levels, listed in the same order as the criteria. If alts is None, we use the entire item space; that is, the set of all possible items.
  • asker -- A callable object f(a, b) that returns a Relation (other than IC) for a and b; greater elements represent greater preference. Use vda() if the asker is synchronous and avda() if it's asynchronous.
  • find_best -- An integer. If set, Artiruno will aim to identify the top find_best items and stop there. Otherwise, Artiruno will try to compare all the alternatives.
  • max_dev -- The maximum number of criteria on which hypothetical items can deviate from the reference item when asking the user to make choices. It's summed across both items; e.g., max_dev = 5 allows 4 deviant criteria compared to 1 deviant criterion, or 3 compared to 2.
  • allowed_pairs_callback -- Called on allowed_pairs for each iteration of the outermost loop.

A PreorderedSet. Its elements will be a subset of the item space and a superset of alts.

async artiruno.avda(criteria, alts=None, asker=None, find_best=None, max_dev=2, allowed_pairs_callback=<function <lambda>>)

As vda(), but accepts an asynchronous asker.


class artiruno.PreorderedSet(elements=(), relations=(), raw_relations=None)

A set equipped with a preorder. Elements can be added to the set, and the order can be updated to make previously incomparable elements comparable.

  • elements -- An iterable of hashable objects to be ordered.
  • relations -- An iterable of triples (a, b, rel), where a and b are objects in elements, and rel is a Relation. By default, all elements are incomparable to each other.
  • raw_relations -- Used internally.

Add x to the set; a no-op if it's already there. x is initially incomparable to all other elements.

cmp(a, b)

Return the Relation between a and b.


Return a copy of the object.

extreme(n, among=None, bottom=False)

Return the top-n subset (or bottom-n subset, if bottom is true) of the items in the set among, or of the whole set if among is not provided.

We define the top-n subset of a preordered set S to be the set of all elements x ∈ S such that

  • x is comparable to all elements of S, and
  • there are at most n - 1 distinct elements a ∈ S such that a > x.

We define the bottom-n subset similarly, with the inequality in the other direction. Notice that the top-n subset may contain more or less than n items.


Return a new PreorderedSet that contains only the requested elements.

graph(namer=<class 'str'>)

Return the set represented as a graphviz.Source object. Requires the Python package graphviz. namer should be a callback that returns a name for an element, as a string.

learn(a, b, rel)

Update the ordering such that a and b are related by rel, a Relation other than IC. Any other relations that can be inferred by transitivity will be added to the set.

Return a list of pairs that were actually updated. If the relation between a and b was already known, the list will be empty. If we made inferences for other pairs, these pairs will be included in the list.

Raise ContradictionError if the new relation isn't consistent with the preexisting relations (other than incomparability).


Return all the maxima among the items in among, or the whole set if among is not provided. The maxima are defined as the top-1 subset, per extreme().


As maxes, but for minima.

summary(namer=<class 'str'>)

Describe all the relations with a string like "A<B C<D E=F". namer should be a callback that returns a name for an element, as a string.

exception artiruno.ContradictionError(k, was, claimed)

Represents an attempt to build an inconsistent order.


class artiruno.Relation(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

An enum.Enum representing the order relation between two objects.

EQ = 0

Equal or equivalent to

GT = 1

Greater than

IC = None

Incomparable (true relation not yet known)

LT = -1

Less than


LT and GT are true. IC and EQ are false.


Swap LT and GT. Return IC or EQ unchanged.

classmethod cmp(a, b)

Return the Relation between a and b corresponding to Python's built-in comparison operators.

Utility functions


Shortcut for itertools.combinations(x, 2).

artiruno.cmp(a, b)

As Python 2's cmp().


Arfer, K. B. (2024). Artiruno: A free-software tool for multi-criteria decision-making with verbal decision analysis. Journal of Multi-Criteria Decision Analysis, 31. doi:10.1002/mcda.1827. Retrieved from http://arfer.net/projects/artiruno/paper

Ashikhmin, I., & Furems, E. (2005). UniComBOS—Intelligent decision support system for multi-criteria comparison and choice. Journal of Multi-Criteria Decision Analysis, 13, 147–157. doi:10.1002/mcda.380

Larichev, O. I., & Moshkovich, H. M. (1995). ZAPROS-LM—A method and system for ordering multiattribute alternatives. European Journal of Operational Research, 82, 503–521. doi:10.1016/0377-2217(93)E0143-L

Moshkovich, H. M., & Mechitov, A. I. (2018). Selection of a faculty member in academia: A case for verbal decision analysis. International Journal of Business and Systems Research, 12, 343–363. doi:10.1504/IJBSR.2018.10011350