Pith. sign in

REVIEW 2 major objections 5 minor 19 references

Interactive Collaborative Exploration using Incomplete Contexts

T0 review · 2 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Pith's one-line read The paper extends attribute exploration for incomplete knowledge to multiple experts, formalizes expert knowledge, interaction, collaboration strategies, and an order comparing strategies by information completeness.

desk verdict Solid formalization of collaborative attribute exploration with incomplete knowledge; central claim holds under the stated non-conflict assumption, and the paper deserves a serious referee. read the letter →

arxiv 1908.08740 v2 pith:FA3ZQZUV submitted 2019-08-23 cs.AI cs.HCcs.LG

classification cs.AIcs.HCcs.LG
keywords FormalConceptAnalysisattributeexplorationincompletecontextsthree-valuedcollaborationstrategyexpertknowledgeinformationorderacquisition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper extends attribute exploration, a formal method for uncovering dependencies among attributes by questioning a domain expert, so that several experts with incomplete knowledge can explore the same domain cooperatively. It models an expert as an incomplete context of example objects plus a set of attribute implications the expert knows are valid in the unknown universe, and it defines an information order that says when one expert's knowledge is at least as complete as another's. A collaboration strategy is formalized as an algorithm that answers the exploration's questions in a way consistent with the universe, using the group of experts. The paper proves that the strategy built from the combined knowledge of all experts yields the maximal obtainable knowledge, and it defines an order comparing the results of different strategies by information completeness. This matters because previously the theory of attribute exploration with incomplete knowledge only supported a single expert.

What carries the argument

The load-bearing structure is the incomplete context $K=(G,M,\{×,o,?\},I)$, whose incidence values carry two orders: a trueness order $o<?<×$ and an information order $?<×$ and $?<o$. The information order on values extends componentwise to incomplete contexts, and a generalized version compares contexts with different object sets; the resulting supremum and infimum operators $\vee_g$ and $\wedge_g$ combine expert example knowledge. Implication knowledge is closed under Armstrong-style consequence, and the product of the example-context lattice and the implication lattice gives a lattice of expert knowledge. Collaboration strategies are defined as algorithms that take an implication question $A\to B$ and a group of experts and return an answer consistent with the universe, satisfying three conditions that prevent accepting invalid implications or giving non-universe counterexamples. An information-completeness order on exploration results, as elements of the same product lattice, is what makes strategies comparable.

What would settle it

Construct a universe $K_U$ and two experts where expert 1 knows $I(g,m)=×$ and expert 2 knows $I(g,m)=o$ for some object $g$ and attribute $m$, while both still satisfy the other conditions. Then the generalized supremum $K_1\vee_g K_2$ is undefined, Lemma 4.20 fails, and any collaboration strategy built on that combination can return an answer that is not a sub-context of the universe. Alternatively, let one expert assert an implication that is actually false in $K_U$; the definition of expert interaction would then not be satisfied, so soundness no longer follows.

Watch

Extended reading notes

Core claim

The central claim is that attribute exploration for incomplete knowledge can be lifted from one expert to a group of experts while preserving soundness. The paper formalizes expert knowledge as a pair consisting of a three-valued context of examples that is a sub-context of the universe and a set of implications valid in the universe; it formalizes interaction as a function returning true, false with counterexamples, or unknown with the attributes still in doubt. A collaboration strategy is any algorithm that maps a question and a group of experts to such an answer under three consistency conditions. The main theorem states that the maximal expert, obtained as the supremum of the group's knowledge in the generalized information order, has maximal knowledge, and attribute exploration using that expert yields the maximum knowledge obtainable from the group. The paper further exhibits broadcast, iterative, and random-selection strategies and shows that accepting one valid implication can require up to $|E|\cdot(|M|-1)$ expert interactions in the worst case.

Load-bearing premise

The framework assumes every expert is reliable and non-conflicting: their examples are partial views of the same true universe, the implications they assert really hold there, and no two experts assign opposite values to the same object and attribute.

Editorial extensions

If this is right

  • Any collaboration strategy meeting the paper's definition is sound: it never accepts an implication that is invalid in the universe and never rejects one with an object outside the universe.
  • The maximal-knowledge strategy $\varphi_{\max}$ gives a benchmark: its exploration result is the most complete result any strategy can obtain from the same group of experts.
  • Broadcast and iterative strategies are expected by the paper to yield approximately the same implication knowledge, but the iterative strategy uses fewer interactions per question, while the broadcast strategy collects more counterexamples.
  • There are universes and expert groups where an implication can be accepted only after consulting experts repeatedly, with a worst-case interaction count of $|E|\cdot(|M|-1)$ per accepted implication.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One extension the paper leaves open is dropping the reliability assumption; a natural test is to replace the supremum by a conflict-resolution rule and check whether the maximal-knowledge result still holds on a finite constructed universe.
  • The interaction-complexity bound suggests that parallelizing question generation, as in the cited parallel exploration work, will not remove the sequential cost of checking distributed implications; this trade-off can be measured by counting oracle calls per query.
  • A concrete next step is to design a scalar metric on the product lattice of examples and implications, as the paper suggests, and to validate it on the Olympic-disciplines running example by comparing the three strategies.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper develops a theoretical framework for collaborative attribute exploration with multiple experts who have incomplete knowledge of a domain. It formalizes expert knowledge as a pair (KE, Cons(LE)) consisting of an incomplete context of partially known objects and a set of attribute implications known to be valid in the universe; introduces a generalized information order, infimum, and supremum for incomplete contexts on possibly different object sets; defines expert interaction and collaboration strategies; proves that the supremum of a group of experts exists and that the maximum-knowledge strategy achieves the maximal obtainable knowledge; and discusses several concrete strategies (single-expert, ignorant, maximum-knowledge, broadcast, iterative, random-selection) together with an interaction-complexity lower bound. A running example with three experts exploring the Olympic sports disciplines illustrates the framework.

Significance. If the framework holds, it provides a first formal foundation for collaborative attribute exploration with incomplete knowledge, a problem that the paper correctly identifies as untreated in the FCA literature. The definitions are coherent, the proofs of Lemma 4.9, Lemma 4.20, and Theorem 4.25 are elementary and correct, and the paper is careful to state its key idealization—experts are reliable and non-conflicting—so that the results are honest conditionals. The running example is detailed and helpful. The main limitations are that the contribution is largely definitional rather than theorem-driven, and that the proposed order for comparing exploration results is not fully extended to exploration outputs containing fictitious counterexamples (see major comment 2).

major comments (2)
  1. [Section 4.5 (Algorithm 1, Remark 4.30, Corollary 4.31)] Algorithm 1 makes a single pass over the group of experts and never updates the premise A with the attributes Y collected during the pass. In contrast, Remark 4.30 and Corollary 4.31 argue about repeated questioning with refined premises: after learning a0→a1 from the first expert, the question is changed to a0a1→a1...an before consulting the remaining experts. Under Algorithm 1 as written, the scenario of Remark 4.30 (two experts with L1={a0→a1} and L2={a1→a2}) would produce (unknown,{a2}) for the question a0→a1a2, even though the combined knowledge entails the implication; the lower bound of Corollary 4.31 therefore does not apply to the strategy defined by Algorithm 1. Please reconcile the definitions: either implement the repeated-questioning loop in Algorithm 1, or restrict Remark 4.30 and Corollary 4.31 to a different explicitly defined strategy, and check the running example (Example 6.1) for consistency.
  2. [Section 4.6 and Section 5] The claim that "the result of an exploration is an element of the product lattice of implications and examples and can be compared in the same way as expert knowledge" is not compatible with the definitions, because exploration results contain fictitious counterexamples (Fact 3.28 and Example 6.1) whose objects are not in the universe G, while the generalized information order ≤g is defined only for contexts with object sets G1,G2⊆G. Hence the exploration result, as presented, is not in the carrier set of the lattice over which ≤g is defined. The information-completeness comparison therefore needs an additional step—for example, comparing only the real counterexamples and treating fictitious objects as part of the implication output, or extending ≤g to contexts with fictitious objects.
minor comments (5)
  1. [Abstract] The abstract contains a grammatical error: "we to develop" should be "we develop".
  2. [Lemma 4.9 proof] In part 1 ('⇒'), the proof asserts that for all (g,m) in G1×M, I1(g,m)=I2(g,m), which is too strong; from K1≤gK2 one only has I1(g,m)≤I2(g,m). The conclusion K1∧gK2=K1 is still correct because the component-wise infimum of ? and × (or ? and o) is ?, but the displayed equality should be replaced with the component-wise infimum identity. In part 2 ('⇒'), "G1≤G2" should read "G1⊆G2".
  3. [Definition 4.21] In property 2, the quantifier "∀g∈G" reuses G both for the universe object set and for the object set of the returned counterexample context K; as written, it would require every object of the universe to be a counterexample. The condition should quantify over the object set of K (or use a distinct symbol such as G_K).
  4. [Throughout] There are several typographical errors: "ficticious" for "fictitious" (multiple occurrences), "dependant" for "dependent", "knowledgable" for "knowledgeable", and "the the universe" in Definition 4.1. These should be corrected.
  5. [Section 4.6] The ranking of the broadcast, iterative, and random-selection strategies is stated informally ("We presume"). Since this is a theoretical paper, the comparison would be more precise if stated as explicit open questions or conjectures with the exact sense of "about the same obtained knowledge" clarified.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: central claims are explicit formal definitions plus external prior results, not predictions fitted to inputs.

full rationale

The paper's derivation chain is a formal-theoretic construction rather than an empirical prediction. Expert knowledge is explicitly defined in Definition 4.2 as a pair (KE, Cons(LE)) with KE <= KU|GE and LE subset Imp(KU). The combination operations in Definitions 4.7, 4.8, and 4.15 are component-wise orders and lattice operations on those defined objects, and Lemma 4.9 proves the lattice property from the definition of S, not from any fitted parameter. The maximal-expert claim in Theorem 4.25 follows from the definition of the supremum and from Fact 3.28, which is imported from the prior work of Burmeister and Holzer; that cited theorem is parameter-free, states its assumptions (expert answers consistent with the universe), and does not include the multi-expert result as an input. The paper explicitly excludes contradictory knowledge in Section 1 ('we neither consider ... contradictory knowledge') and in Remark 4.5, so the reliance on non-conflicting experts is a declared scope restriction, not a hidden equivalence. The only passage resembling self-citation, the pointer to Stumme's earlier work on exceptions to attribute implications in Section 3.5, is a passing remark and is not load-bearing for the central framework. No equation is defined in terms of the quantity it is said to derive, no parameter is fitted and then renamed a prediction, and no uniqueness or existence claim rests on a self-citation chain. The honest finding is therefore no circularity.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

There are no free parameters or invented physical entities. The framework's load-bearing assumptions are the idealizations that experts are correct, non-contradicting, and that the domain is a finite formal context; these are stated explicitly and are reasonable for a first theoretical treatment.

assumptions (4)
  • domain assumption The universe KU can be represented as a formal context (G,M,I), with finite attribute set M.
    Definition 4.1 and Section 3.5; attribute exploration requires finite attributes and a well-defined universe of objects.
  • domain assumption Each expert's knowledge is correct but incomplete: KE <= KU|GE and LE subset of Imp(KU).
    Definition 4.2; the framework excludes unreliable experts, and all lemmas about combining knowledge rely on this consistency.
  • domain assumption Experts never hold contradictory knowledge; any two contexts can be combined without conflict.
    Definition 4.8, Lemma 4.20, and the introduction state that contradictory knowledge is out of scope; the supremum of expert contexts only exists without conflicts.
  • standard math The known results on incomplete contexts and attribute exploration (Kripke/Kleene equivalence, Cons(Imp(K))=Imp(K), and the maximal-information property of single-expert exploration) are accepted from cited literature.
    Theorems 3.24, Lemmas 3.25 and 3.26, and Fact 3.28 are taken from [2,7,9,11] as background; the present framework is built on top of them.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Interactive Collaborative Exploration using Incomplete Contexts." pith.science (2026). https://pith.science/paper/FA3ZQZUV

@misc{pith2026190808740,
  author       = {Pith},
  title        = {Pith review of: Interactive Collaborative Exploration using Incomplete Contexts},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FA3ZQZUV}},
  note         = {Machine review of arXiv:1908.08740}
}
read the original abstract

A well-known knowledge acquisition method in the field of Formal Concept Analysis (FCA) is attribute exploration. It is used to reveal dependencies in a set of attributes with help of a domain expert. In most applications no single expert is capable (time- and knowledge-wise) of exploring the knowledge domain alone. However, there is up to now no theory that models the interaction of multiple experts for the task of attribute exploration with incomplete knowledge. To this end, we to develop a theoretical framework that allows multiple experts to explore domains together. We use a representation of incomplete knowledge as three-valued contexts. We then adapt the corresponding version of attribute exploration to fit the setting of multiple experts. We suggest formalizations for key components like expert knowledge, interaction and collaboration strategy. In particular, we define an order that allows to compare the results of different exploration strategies on the same task with respect to their information completeness. Furthermore we discuss other ways of comparing collaboration strategies and suggest avenues for future research.

Figures

Figures reproduced from arXiv: 1908.08740 by the authors.

Figure 1
Figure 1. Example of two representations of a formal context about the sports [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Example of an incomplete contexts with attributes a) [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. a), and we can order them according to the amount of information they represent, i.e., ?<× and ?<o (information order, see Figure 3b). In the latter case the values × and o are incomparable. × ? o (a) trueness order (Kleene-Logic) × ? o (b) information order (Kripke-Semantics) [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Visualization of exploration with one expert [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: Visualization of attribute exploration with multiple experts [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: An example of a context where implicational knowledge can be adversely [PITH_FULL_IMAGE:figures/full_fig_p027_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 19 canonical work pages

  1. [1]

    Dependency Structures of Data Base Relation- ships

    William Ward Armstrong. “Dependency Structures of Data Base Relation- ships.” In:IFIP Congress. 1974, pp. 580–583

  2. [2]

    On the Treatment of Incomplete Knowledge in Formal Concept Analysis

    P. Burmeister and R. Holzer. “On the Treatment of Incomplete Knowledge in Formal Concept Analysis.” In:ICCS. Ed. by B. Ganter and G. W. Mineau. Vol. 1867. Lecture Notes in Computer Science. Springer, 2000, pp. 385–398

  3. [3]

    Merkmalimplikationen bei unvollständigem Wissen

    Peter Burmeister. “Merkmalimplikationen bei unvollständigem Wissen.” In: ArbeitstagungBegriffsanalyseundKünstlicheIntelligenz .Ed.byWilfriedLex. Informatik-Bericht 89/3. Clausthal-Zellerfeld, 1991, pp. 15–46

  4. [4]

    Two basic algorithms in concept analysis (Preprint 831)

    B Ganter. “Two basic algorithms in concept analysis (Preprint 831).” In: Darmstadt: Technische Hochschule(1984)

  5. [5]

    Ganter and S

    B. Ganter and S. Obiedkov.Conceptual Exploration. Springer, 2016

  6. [6]

    Attribute exploration with background knowledge

    Bernhard Ganter. “Attribute exploration with background knowledge.” In: Theoretical Computer Science217.2 (1999), pp. 215–233

  7. [7]

    Berlin/Heidelberg: Springer-Verlag, 1999

    Bernhard Ganter and Rudolf Wille.Formal Concept Analysis: Mathematical Foundations. Berlin/Heidelberg: Springer-Verlag, 1999

  8. [8]

    Towards Collaborative Conceptual Explo- ration

    Tom Hanika and Jens Zumbrägel. “Towards Collaborative Conceptual Explo- ration.” In:ICCS.Ed.byP.Chapman,D.Endres,andN.Pernelle.Vol.10872. Lecture Notes in Computer Science. Springer, 2018, pp. 120–134

Show all 19 references
  1. [9]

    Knowledge acquisition under incomplete knowledge using methods from formal concept analysis: Part I

    Richard Holzer. “Knowledge acquisition under incomplete knowledge using methods from formal concept analysis: Part I.” In:Fundamenta Informaticae 63.1 (2004), pp. 17–39. REFERENCES 31

  2. [10]

    Knowledge acquisition under incomplete knowledge using methodsfromformalconceptanalysis:PartII

    Richard Holzer. “Knowledge acquisition under incomplete knowledge using methodsfromformalconceptanalysis:PartII.” In: FundamentaInformaticae 63.1 (2004), pp. 41–63

  3. [11]

    Shaker, 2001

    Richard Holzer.Methoden der formalen Begriffsanalyse bei der Behandlung unvollständigen Wissens. Shaker, 2001. 215 pp

  4. [12]

    Introduction to Metamathematics

    Stephen Cole Kleene. “Introduction to Metamathematics.” In:Journal of Symbolic Logic33.2 (1968), pp. 290–291

  5. [13]

    ParallelAttributeExploration

    FrancescoKriegel.“ParallelAttributeExploration.” In:ICCS.Ed.byOllivier Haemmerlé, Gem Stapleton, and Catherine Faron-Zucker. Vol. 9717. Lecture Notes in Computer Science. Springer, 2016, pp. 91–106

  6. [14]

    Semantical Analysis of Modal Logic I Normal Modal Proposi- tional Calculi

    S.A. Kripke. “Semantical Analysis of Modal Logic I Normal Modal Proposi- tional Calculi.” In:Mathematical Logic Quarterly9.5-6 (1963), 67–96

  7. [15]

    Collaborative conceptual exploration as a tool for crowdsourcing domain ontologies

    S. Obiedkov and N. Romashkin. “Collaborative conceptual exploration as a tool for crowdsourcing domain ontologies.” In: vol. Vol-1552. 2015, pp. 58–70

  8. [16]

    Modal Logic for Evaluating Formulas in Incomplete Contexts

    Sergei A. Obiedkov. “Modal Logic for Evaluating Formulas in Incomplete Contexts.” In:Proceedingsofthe10thInternationalConferenceonConceptual Structures (ICCS 2002). Ed. by Uta Priss, Dan Corbett, and Galia Angelova. Vol. 2393. Lecture Notes in Computer Science. Springer, 2002,...

  9. [17]

    InformationGapsasCommunicationNeeds:ANewSemantic Foundation for Some Non-Classical Logics

    PieroPagliani.“InformationGapsasCommunicationNeeds:ANewSemantic Foundation for Some Non-Classical Logics.” In:Journal of Logic, Language and Information6.1 (Jan. 1997), pp. 63–99

  10. [18]

    Attribute Exploration with Background Implications and Exceptions

    Gerd Stumme. “Attribute Exploration with Background Implications and Exceptions.” In:Data Analysis and Information Systems. Statistical and Conceptual approaches. Proc. GfKl’95. Studies in Classification, Data Anal- ysis, and Knowledge Organization 7. Ed. by H.-H. Bock and W. P...

  11. [19]

    Restructuring lattice theory: An approach based on hierarchies of concepts

    R. Wille. “Restructuring lattice theory: An approach based on hierarchies of concepts.” In:Ordered Sets. Ed. by I. Rival. Reidel, Dordrecht-Boston, 1982, pp. 445–470. 32 REFERENCES 6 Appendix 6.1 Disciplines of the Summer Olympic Games 2020 Context ≥ 5 events ≥ 10 events ≥ 20 ...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.