DAST Model: Deciding About Semantic Complexity of a Text

Mohammad Izadi; MohammadReza Besharati

REVIEW 4 major objections 5 minor 5 references

DAST Model: Deciding About Semantic Complexity of a Text

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Semantic complexity can be decided by building a lattice of intuitions, the DAST model claims, and human judgments agree with it in large experiments.

desk verdict The formal framework is novel and the experiments are large, but DASTEX is a human expert count, so the claim that DAST 'decides' semantic complexity is not supported as stated. read the letter →

arxiv 1908.09080 v5 pith:CWPL7RIA submitted 2019-08-24 cs.CL cs.AI

MohammadReza Besharati , Mohammad Izadi This is my paper · ORCID

classification cs.CLcs.AI

keywords semanticcomplexityintuitionisticsemanticslatticetexthumanjudgmentreadabilityassessmentDASTtheories

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to establish that semantic complexity—how much meaning-related work a text demands—can be decided by a formal, intuitionistic model rather than by word counts, syntax, or readability formulas. Meaning is modeled as a lattice grown from basic intuitions by symbolic rules, and a text's semantic complexity is a computed value on that lattice. The paper reports that DAST's complexity rankings track human comparative judgments: in a mutation experiment with 3,198 participants, DAST agreed with the majority human choice on the overall most complex sentence in 70% of cases, against a 20% random baseline, and across 80 sentences vote-values correlated with DAST's relative complexity values at $R^2=0.83$. A simpler corpus-facing index, DASTEX (the number of semantic theories a text invokes), is claimed to have distinction power close to fixation time and word count on a readability corpus. If correct, the model gives a principled bridge from theories of meaning to computable text-complexity scores.

What carries the argument

The load-bearing object is the semantic lattice: a directed structure of symbol strings produced by applying Semantic Logic rules whenever a rule's left side is present in working memory. Rules give the lattice its edges, and a valuation function $V$ assigns each node a number based on its predecessors; the complexity calculation algorithm $CA$ then turns the valued lattice into a number. Semantic items with many combined inputs receive higher values, and the whole sentence gets an overall complexity as the distance of its semantic point from the origin in an $n$-dimensional semantic space whose axes are semantic items. The simpler DASTEX index—counting a text's involving semantic theories—is the version of the machinery used for corpus evaluation, and it is explicitly a first-level estimation rather than the full lattice calculation.

What would settle it

Have several independent annotators enumerate the involving semantic theories for the 32 Scanpath paragraphs and the 80 Persian sentences; if their DASTEX counts disagree substantially, or if DASTEX fails to correlate with readers' complexity judgments and fixation times on a new set of texts spanning the same genres, the claim that DAST decides semantic complexity would be falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that a text's semantic complexity is not a hidden quality but a computable object: for a given text, a set of principal intuitions, a Semantic Logic of derivation rules, and a valuation scheme, the construction of the semantic lattice and a calculation on that lattice yields a definite complexity value. The 6-tuple $CM=(T,P,SA,L,V,CA)$ formalizes this, and an overall sentence-level value is the distance of the text's semantic point from the origin of the semantic space spanned by its semantic items. The authors argue the model decides about semantic complexity in the sense that its judgments reproduce the comparative judgments of human readers: majority human choices in three mutation experiments matched DAST 64–78% of the time, the largest experiment showing 70% overall precision against a 20% random baseline, and vote-values for 80 sentences correlated linearly with DAST's relative complexity values at $R^2=0.83$. They further claim the pattern of human deviations from DAST follows an exponential curve, which they model as a Markovian process of multi-step common-sense reasoning, and that consensus with the axiomatized Semantic Logic has a sigmoid-like triggering effect on agreement with DAST.

Load-bearing premise

The load-bearing premise is that a human expert can reliably enumerate all semantic theories a text involves, and that this count—or the expert-built lattice—faithfully measures semantic complexity; in the corpus study DASTEX was computed manually by the authors, so the central quantity is not yet an automated, reproducible measurement.

Editorial extensions

If this is right

If DAST is correct, semantic complexity can be measured without relying on sentence length, vocabulary lists, or parse-tree features, so readability assessment could be extended to meaning-level difficulty.
A reusable Semantic Logic rule base could amortize the expert effort of building lattices, making DAST-style analysis practical for new domains once such a base exists.
The Markovian-noise model implies that human disagreement with a common-sense complexity ordering should fall exponentially with the number of deviated comparison steps, a quantitative prediction that can be checked on new judgment data.
The vote-value correlation ($R^2=0.83$) supports treating DAST's relative complexity values as an estimator of group complexity votes, with applications in text selection and simplification.
On the Scanpath corpus, DASTEX's difficulty ratio sits in the same cluster as fixation time and word count, suggesting a semantic measure can mimic both an objective and a subjective readability signal.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper leaves implicit that if DASTEX enumeration were automated—say, by a classifier trained on expert-annotated semantic theories—the index would become reproducible and could be tested on much larger corpora without author intervention.
The genre split in the corpus results suggests DASTEX's relation to reading effort is not uniform across domains; one testable consequence is that a genre-aware semantic index, rather than a single global formula, may be the right target for automatic readability systems.
The sigmoid consensus-agreement curve hints at a threshold effect: below some consensus level, readers may effectively use different semantic logics, so DAST's complexity ordering would only track the majority past that threshold.
The lattice formalism also yields local node values that are never aggregated; those values could be tested directly as predictors of word- or phrase-level reading times, a prediction the paper does not make.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know this paper claims more than it delivers, but it isn't empty. The new thing is a formal framework treating semantic complexity as a construction over lattices of intuitions, plus a simple index (DASTEX) defined as the number of semantic theories involved in a text. That specific formalization and the Markovian model of multi-step reasoning seem genuinely new. The paper also reports unusually large human-judgment experiments (over 7,000 participants across scenarios). That scale is real.

What it does well: the 6-tuple formal system (Definition 1) is an honest attempt to make the components explicit, and the authors are transparent about needing to hand-craft Semantic Logic rules per text — they say one logic took about 10 man-hours. The Java-automated deduction for SUS-2 is a concrete reproducibility point for that one example.

The soft spots are not minor. The central measurement, DASTEX, is described in Definition 6 and then in the corpus evaluation is actually computed by a human expert counting semantic theories. No operational rule is given for identifying a theory, so the numbers behind the headline R²=0.83 and the corpus comparisons are human annotations dressed in formal notation. On top of that, the Semantic Logic axioms in the mutation experiment are hand-authored to derive exactly the semantic loads (Wonder, Engagement) that participants are asked to judge, and the participants are asked to agree with those axioms — so part of the agreement is baked in. The corpus evaluation is also fragile: Hypothesis 2 fails on all 16 data points before the authors split by genre and drop outliers to get R²=0.98/0.96. That is post-hoc analysis, not confirmation.

Even so, I would not dismiss the paper. The mutation experiment against a 20% random baseline is a genuine signal, even if only on one sentence family. The framework is a plausible starting point for future automated work. The right verdict is 'major revision': require the authors to either automate DASTEX or provide a clear annotation protocol with inter-annotator agreement, and to report the failed hypothesis without post-hoc filtering as the primary result. It deserves a serious referee, but it is not ready as is.

Referee Report

4 major / 5 minor

Summary. The paper introduces the DAST model, a framework for measuring the semantic complexity of text. The model is built on an intuitionistic view of semantics: a text's meaning is represented as a lattice of symbols generated by a set of rules, and semantic complexity is defined as a value computed on this lattice. The authors give a set-theoretic formal definition of a semantic complexity tuple CM=(T, P, SA, L, V, CA), define a semantic space and semantic point, and propose DASTEX, a 'first level estimation' of complexity as the number of involving semantic theories. The evaluation consists of a worked example, three human-judgment experiments on one target sentence and its mutants, sixteen experiments on Persian poem sentences, and a corpus-based comparison against the Scanpath Complexity dataset using difficulty ratios. The authors report a 70% overall precision against a 20% random baseline in the mutation experiment, R^2=0.83 for the correlation between vote-values and DAST relative-values, and R^2=0.98/0.96 for genre-split, outlier-excluded regressions in the corpus evaluation. They also propose a Markovian model for common-sense multi-step reasoning and a gate mechanism for semantic complexity agreement.

Significance. If the central claim were supported—that a formal, automated procedure can decide semantic complexity of text—the work would contribute a conceptually distinctive approach to text complexity, drawing on intuitionistic logic and lattice theory rather than surface/syntactic features. The paper has some genuine positive aspects: it reports a very large human-judgment collection (more than 12,000 judgments in the first scenario, 15,000+ overall), supplies a dataset DOI, and includes a Java-implemented automatic deduction for one sentence (SUS-2). These are useful resources. However, the significance is not established as written, because the core quantity DASTEX is not computed by the model but by a human expert, and the formal components of the 6-tuple are left unspecified, so the reported correlations do not demonstrate that DAST itself decides semantic complexity. The evaluation also relies on post hoc splits and outlier removal to obtain its headline corpus results. The paper is better read as a report of an expert-assisted framework with preliminary correlations, not as a validated decision procedure.

major comments (4)

[Section 3.4, Definition 1] The formal system CM=(T,P,SA,L,V,CA) leaves SA, V, and CA as abstract functions without concrete definitions or instantiations, so the paper does not establish that DAST itself decides semantic complexity; the decision is effectively delegated to whoever supplies these components. The claim in the Abstract and Section 6.2 that 'DAST model is capable of deciding about semantic complexity' is therefore not supported by the formal apparatus.
[Section 6.3.1 and Definition 6] DASTEX is computed by a human expert, not by the model: Section 6.3.1 states that 'a human expert analyzed the paragraphs to enumerate involving semantic theories,' and Section 6.1 reports that eliciting one Semantic Logic took about 10 man-hours. Consequently, the reported R2=0.83 in Figure 18 and the difficulty-ratio comparisons in Table 5 are correlations between human judgments and human-assigned indices, not between human judgments and outputs of the DAST decision procedure. This undermines the central claim of automatic semantic complexity measurement and raises a circularity concern, since both sides of the comparison originate in human judgment.
[Section 6.3.1.1, Figure 20] Hypothesis 2 is initially a null result on all 16 data points; the supporting regressions appear only after splitting the data by genre into two 8-point classes and, for Class 2, excluding one outlier, yielding R2=0.98 and R2=0.96. With only 8 points per class, a post hoc split combined with outlier exclusion cannot support the claim that 'the general claim of Hypothesis 2 has been supported by the results of this experiment.' The degrees of freedom used in this analysis are not reported or corrected, and the result is fragile enough that the corpus evaluation does not validate DAST as a general measure.
[Section 6.2.1] In the mutation experiment, the Semantic Logic for SUS-2 was hand-crafted so that the deduction yields semantic loads such as Wonder and Engagement, and participants were first asked to agree with the axioms before making judgments. The reported 70% precision against a 20% random baseline therefore partly re-states the axioms; the experiment does not independently validate the model's complexity ordering, because the axioms already encode the expected conclusion. A more convincing test would fix the Semantic Logic in advance or apply it to a held-out set of sentences without prior participant agreement.

minor comments (5)

[Section 2] The Flesch-Kincaid formula is presented with ellipses and garbled symbols (RGFK = 0.39... + 11.8...); the formula should be typeset correctly for reproducibility.
[Section 6.2.2] The text states that 'Precision for Overall Result has a Mean 58%,' but Table 3 reports Avg.=61% with STDEV=13; the text and table are inconsistent and should be reconciled.
[Section 6.3.1, Definition 9] The formula for Difficulty Ratio (DR) is missing from the manuscript; only the verbal definition appears, so the reader cannot verify the computed DR values.
[Section 6.2.1, Figure 15] The Markovian model introduces transition probabilities αi and βi without defining them in the text; these symbols are central to the proposed model and require explicit definitions and constraints.
[Abstract and Section 6.3.1.3] The Abstract claims that DAST 'competes with other methods by less error percentage,' but the evaluation reports error percentages of DASTEX relative to three reference measures (Fixation-Time, Word-Count, Readability-Level), not a head-to-head comparison against other readability methods; the claim should be restated accordingly.

Circularity Check

1 steps flagged · score 6.0 of 10

Corpus evaluation's DASTEX is a human expert's theory count, not a DAST-model output; the reported correlations therefore validate human judgment against itself.

other [Section 6.3.1.1 (Corpus Based Evaluation; Definition 6 in Section 5)]
"We calculate the DASTEX-DR values for every of 16 paragraph-pairs by running our experiment. For this, a human expert analyzed the paragraphs to enumerate involving semantic theories."

Definition 6 defines DASTEX as 'number of involving semantic theories', but the paper supplies no operational rule for identifying a semantic theory. In the corpus evaluation, the DASTEX values are not outputs of the Definition 1 six-tuple or of the Java deduction described in Section 6.1; they are counts produced by a human expert. The later correlations between DASTEX-DR and eye-fixation or readability-level judgments are therefore correlations between one set of human judgments (the expert's theory counts, labeled 'DAST') and another set of human judgments (participants' reading effort). The claimed 'DAST method' prediction reduces by construction to the expert's manual enumeration, so it cannot independently validate DAST as a decision procedure for semantic complexity.

full rationale

The clearest circular step is in the corpus evaluation. DASTEX is defined in Definition 6 as the number of involving semantic theories, and Section 6.3.1.1 states that a human expert performed the enumeration. Thus the 'DAST-measured semantic-complexity estimations' used in the later comparisons are manually produced expert annotations, not outputs of the formal CM six-tuple or the automated Java deduction. Comparing these expert counts with eye-fixation durations or readability annotations is human-judgment-versus-human-judgment; it cannot establish that DAST itself 'decides' semantic complexity. This is partial circularity because the predictor variable is the same kind of human semantic judgment the model claims to replace. The mutation experiment is less clearly circular: the Semantic Logic-2 is hand-crafted, and the participants were asked about axiom agreement, but the Java deduction is an independent computation and the comparison is between the rules and participant judgments. However, the paper's headline claim is broader than that one experiment, and the main corpus/correlation evidence depends on the expert-supplied DASTEX. The under-specification of P, SA, V, and CA in Definition 1 aggravates the problem by allowing the expert's choices to enter without constraint, but the circularity itself is the substitution of expert enumeration for model output. Score 6 reflects partial circularity, not full equivalence: the mutation experiment adds some independent computational content, while the corpus-based 'predictions' reduce to human expert input.

Assumptions & free parameters 4 free parameters · 6 assumptions · 3 invented entities

The central computation depends on hand-authored Semantic Logic rules and a manually counted DASTEX index. These choices, together with post-hoc data splits and outlier exclusions, are the main contributors to the reported results. No automatic procedure is provided to derive rules or count theories, so the model's predictive content is carried by human judgment.

free parameters (4)

Semantic Logic rules per text = hand-authored
The complexity output for each sentence is determined by rules the authors wrote manually; no automatic procedure is supplied.
Lattice value configuration schema = unspecified
In Section 4, node values are assigned by a schema shown only in a figure, without a closed-form definition.
Outlier exclusion = 1 point dropped in Class2 regression; 2 points dropped in figure 19
Reported R2 values (0.96 for Class2, extremum extrapolation) depend on excluding specific data points after inspecting the data.
Genre split = history/literature vs geography/science
The 16 paragraph pairs are split into two classes only after the full 16-point regression failed, a post-hoc grouping.

assumptions (6)

domain assumption Semantics can be represented as a lattice of intuitions
Section 3.2 postulates that meaning is a construction of basic realities and intuitions.
domain assumption Any well-meaning text references a combination of realities and intuitions
Section 3.2 uses this to justify representing text meaning as a lattice.
ad hoc to paper The hand-written Semantic Logic rules are an axiomatization of common sense
Section 6.2.1 reports 88-100% participant agreement, but the rules are authored by the researchers for the specific sentences used in evaluation.
ad hoc to paper Complexity of a sentence is the distance of its semantic point from the origin in semantic space
Definitions 2-4 introduce this linear-algebraic model without independent justification.
ad hoc to paper Number of involving semantic theories is a first-level estimate of semantic complexity
Definition 6 defines DASTEX; the count is performed manually by a human expert.
standard math Brouwer-Heyting-Kolmogorov interpretation for intuitionistic logic
Section 3.1 invokes BHK semantics as the basis for Semantic Logic.

invented entities (3)

Semantic space and semantic point
purpose: To define overall semantic complexity as the Euclidean distance from the origin
Definitions 2-4 create a geometric model with no external falsifiable anchor in the paper.
Gate mechanism for semantic complexity
purpose: To explain the sigmoid-like relation between consensus on axioms and agreement with DAST
Section 6.2.1 proposes a cognitive gate mechanism; no dedicated experiments test it.
Markovian model of common-sense multi-step reasoning
purpose: To explain the exponential drop-off in deviation counts
Section 6.2.1 introduces alpha and beta probabilities without fitting or validating them on held-out data.

how reviews work

0 comments

Cite this review

Pith. "Pith review of DAST Model: Deciding About Semantic Complexity of a Text." pith.science (2026). https://pith.science/paper/CWPL7RIA

@misc{pith2026190809080,
  author       = {Pith},
  title        = {Pith review of: DAST Model: Deciding About Semantic Complexity of a Text},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CWPL7RIA}},
  note         = {Machine review of arXiv:1908.09080}
}

read the original abstract

Measuring text complexity is an essential task in several fields and applications (such as NLP, semantic web, smart education, etc.). The semantic layer of text is more tacit than its syntactic structure and, as a result, calculation of semantic complexity is more difficult than syntactic complexity. While there are famous and powerful academic and commercial syntactic complexity measures, the problem of measuring semantic complexity is still a challenging one. In this paper, we introduce the DAST model, which stands for Deciding About Semantic Complexity of a Text. DAST proposes an intuitionistic approach to semantics that lets us have a well-defined model for the semantics of a text and its complexity: semantic is considered as a lattice of intuitions and, as a result, semantic complexity is defined as the result of a calculation on this lattice. A set theoretic formal definition of semantic complexity, as a 6-tuple formal system, is provided. By using this formal system, a method for measuring semantic complexity is presented. The evaluation of the proposed approach is done by a set of three human-judgment experiments. The results show that DAST model is capable of deciding about semantic complexity of text. Furthermore, the analysis of the results leads us to introduce a Markovian model for the process of common-sense, multiple-steps and semantic-complexity reasoning in people. The results of Experiments demonstrate that our method outperforms the random baseline with improvement in better precision and competes with other methods by less error percentage.

Figures

Figures reproduced from arXiv: 1908.09080 by the authors.

**Figure 2.** Another tool for idea density measurement is calculating average number of senses per word which is reported as a successful feature for readability assessment (Pilán, Volodina, & Johansson, 2014). By Categorization and Classification of words (such as Content Word Classification (Mühlenbock, 2013)), Connectionist and Network Models (such as Semantic Cognition Model (Rogers & McClelland, 2004), WordNet Based Approac… view at source ↗

**Figure 14.** figure 14 [PITH_FULL_IMAGE:figures/full_fig_p024_14.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

5 extracted references · 3 canonical work pages

[1]

Abdel-Fattah, A. M. H., & Krumnack, U. (2013). Creating analogy -based interpretations of blended noun concepts. AAAI Spring Symposium: Creativity and (Early) Cognitive Development. Badgett, B. A. (2010). Toward the development of a model to estimate the readability of credentialing - examination materials. University of Nevada. Besharati, M. R., & Izadi,...

arXiv 2013
[38]

Freitas, A. (2015). Schema-Agnostic Queries for Large -Schema Databases: A Distributional Semantics Approach. National University of Ireland. Freitas, A., Sales, J. E., Handschuh, S., & Curry, E. (2015). How hard is this query? Measuring the semantic complexity of schema -agnostic queries. IWCS 2015 - Proceedings of the 11th International Conference on Co...

work page 2015
[223]

Ralph, M. A. L., Jefferies, E., Patterson, K., & Rogers, T. T. (2016). The neural and computational bases of semantic cognition. Nature Reviews Neuroscience. https://doi.org/10.1038/nrn.2016.150 Robertson, A., Aiello, L. M., & Quercia, D. (2019). The Language of Dialogue Is Complex. Proceedings of the International AAAI Conference on Web and Social Media,...

arXiv 2016
[2007]

https://doi.org/10.1109/COMPSAC.2007.153 Kant, K., & Srinivasan, M

, 673 –675. https://doi.org/10.1109/COMPSAC.2007.153 Kant, K., & Srinivasan, M. M. (1992). Introduction to computer system performance evaluation . McGraw- Hill. Kauchak, D., Mouradi, O., Pentoney, C., & Leroy, G. (2014). Text simplification tools: Using machine learning to discover features that identify difficult text. Proceedings of the 47th Hawaii Int...

work page arXiv 1992
[2009]

https://doi.org/10.2991/agi.2009.28 Haase, C. (2016). Quantify ing Lexico -Semantic Complexity in Academic Writing with Complexana. International Journal of Language and Linguistics, 3(6), 42–50. Hansson, K., B ååth, R., L öhndorf, S., Sahl én, B., & Sikstr öm, S. (2016). Quantifying Semantic Linguistic Maturity in Childre n. Journal of Psycholinguistic R...

work page doi:10.2991/agi.2009.28 2016

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Abdel-Fattah, A. M. H., & Krumnack, U. (2013). Creating analogy -based interpretations of blended noun concepts. AAAI Spring Symposium: Creativity and (Early) Cognitive Development. Badgett, B. A. (2010). Toward the development of a model to estimate the readability of credentialing - examination materials. University of Nevada. Besharati, M. R., & Izadi,...

arXiv 2013

[2] [38]

Freitas, A. (2015). Schema-Agnostic Queries for Large -Schema Databases: A Distributional Semantics Approach. National University of Ireland. Freitas, A., Sales, J. E., Handschuh, S., & Curry, E. (2015). How hard is this query? Measuring the semantic complexity of schema -agnostic queries. IWCS 2015 - Proceedings of the 11th International Conference on Co...

work page 2015

[3] [223]

Ralph, M. A. L., Jefferies, E., Patterson, K., & Rogers, T. T. (2016). The neural and computational bases of semantic cognition. Nature Reviews Neuroscience. https://doi.org/10.1038/nrn.2016.150 Robertson, A., Aiello, L. M., & Quercia, D. (2019). The Language of Dialogue Is Complex. Proceedings of the International AAAI Conference on Web and Social Media,...

arXiv 2016

[4] [2007]

https://doi.org/10.1109/COMPSAC.2007.153 Kant, K., & Srinivasan, M

, 673 –675. https://doi.org/10.1109/COMPSAC.2007.153 Kant, K., & Srinivasan, M. M. (1992). Introduction to computer system performance evaluation . McGraw- Hill. Kauchak, D., Mouradi, O., Pentoney, C., & Leroy, G. (2014). Text simplification tools: Using machine learning to discover features that identify difficult text. Proceedings of the 47th Hawaii Int...

work page arXiv 1992

[5] [2009]

https://doi.org/10.2991/agi.2009.28 Haase, C. (2016). Quantify ing Lexico -Semantic Complexity in Academic Writing with Complexana. International Journal of Language and Linguistics, 3(6), 42–50. Hansson, K., B ååth, R., L öhndorf, S., Sahl én, B., & Sikstr öm, S. (2016). Quantifying Semantic Linguistic Maturity in Childre n. Journal of Psycholinguistic R...

work page doi:10.2991/agi.2009.28 2016