Building the corpus across eight data universes and six tradition buckets while preventing source-family leakage, so cross-cultural recurrence reflects real convergence rather than copying.

title: "Benchmark Corpus and Holdout Plan" created: "2026-06-20" artifact_type: "benchmark_design" status: "draft v0.1" purpose: "Define how to build a corpus for latent graphlet discovery without overfitting, preserving train/validation/test/holdout discipline across cultures, source families, and mechanisms."

Benchmark Corpus and Holdout Plan

0. Core Rule

Treat this like a trading model.

Do not discover and validate on the same corpus.
Do not tune graphlets after seeing holdout results.
Do not count copied sources as independent.
Do not collapse cultures into one symbolic soup.
Do not let a preferred metaphysical hypothesis define the labels.

The first benchmark is not about proving any single theory. It is about testing whether graph structures recur across heterogeneous traditions and cases more than they should under null models.

1. Data Universes

We need several universes, each with different leakage risk.

U1 Ancient / classical religious-text universe
U2 Medieval religious-commentary / demonology universe
U3 Early modern esoteric / occult universe
U4 Modern political cult / coercive-group universe
U5 Modern intelligence / psychological-operations universe
U6 Modern algorithmic / platform / radicalization universe
U7 Clinical / psychological / trauma-language universe
U8 Comparative mythology / anthropology universe

Each universe should have:

source_id
source_family_id
tradition
culture
language
date_range
source_type
access_type
risk_flags
canonical-role mapping status
holdout_status

2. Initial Tradition / Culture Buckets

These are not truth claims. They are sampling strata.

2.1 Abrahamic / Near Eastern

Judaism
Christianity
Islam
Gnosticism
Zoroastrianism
Mesopotamian / Babylonian / Assyrian
Yazidi / related regional traditions

Candidate local concepts:

Satan
ha-satan
Samael
Lilith
shedim
Iblis
shayatin
jinn
waswasa
dajjal
archons
demiurge
Ahriman / Angra Mainyu

2.2 Indian / Buddhist / Himalayan

Hindu traditions
Buddhism
Tantric Buddhism
Jainism
Tibetan / Himalayan demonology
Nepali local traditions

Candidate local concepts:

Mara
asura
rakshasa
preta / hungry ghost
kleshas
avidya
maya
bhuta
pishacha
wrathful deities
obstacle spirits

2.3 Greco-Roman / European

Greek myth
Roman religion
Saturnalia / Carnival / Feast of Fools
Norse / Germanic myth
Celtic fairy / trickster traditions
Early modern witchcraft / demonology

Candidate local concepts:

Saturn
Kronos
Dionysian inversion
trickster
Loki
fae
witchcraft demonology
possession
incubus/succubus

2.4 Indigenous / animist / spirit-contagion systems

Algonquian wetiko traditions
Amazonian spirit/pathology traditions
Siberian shamanic adversary/possession motifs
African and Afro-diasporic possession traditions
Australian Aboriginal sorcery/curse motifs

Candidate local concepts:

wetiko
sorcery intrusion
spirit possession
predatory hunger
curse object
ancestral disorder

2.5 East Asian

Chinese demonology
Daoist spirit pathology
Japanese yōkai / oni / possession
Korean shamanic disorder
Tibetan-Buddhist overlap

Candidate local concepts:

hungry ghost
gui
yaoguai
oni
kitsune possession
yōkai
spirit affliction

2.6 Modern coercive systems

Jonestown / Peoples Temple
SLA
O9A
764
NXIVM
Children of God
Aum Shinrikyo
Scientology-derived splinters
Epstein/Maxwell network
COINTELPRO / MKUltra / intelligence case files
modern online radicalization
algorithmic outrage loops

3. Train / Validation / Test / Holdout Splits

3.1 Split A: Cross-cultural adversary motifs

Train:
  Christianity
  Judaism
  Gnosticism
  Greek/Roman

Validation:
  Islam
  Buddhism
  Hinduism
  Zoroastrianism

Test:
  East Asian
  Indigenous / wetiko
  African diaspora
  Norse / Celtic

Final holdout:
  Modern cults
  online radicalization
  clinical intrusive-thought language
  algorithmic outrage platforms

3.2 Split B: Coercive-group mechanisms

Train:
  Jonestown
  O9A
  764

Validation:
  NXIVM
  Children of God
  Aum Shinrikyo

Test:
  SLA
  Epstein/Maxwell
  Scientology-derived splinters

Final holdout:
  modern online sextortion / radicalization cells
  state-informant disruption cases
  platform-mediated rage networks

3.3 Split C: Ritual inversion / controlled revolt

Train:
  Saturnalia
  Carnival
  Feast of Fools

Validation:
  trickster festivals
  Dionysian rites
  liminal inversion rituals

Test:
  modern protest carnivals
  controlled opposition patterns
  media outrage cycles

Final holdout:
  platform outrage events
  election-year manufactured conflict

3.4 Split D: Legitimacy laundering

Train:
  Reichskonkordat
  Lateran Treaty
  Scofield / Christian Zionism bridge

Validation:
  imperial religious charters
  Cold War religious fronts
  NGO civil-society fronts

Test:
  human-rights language used in war
  corporate moral language
  AI-safety / public-good legitimacy laundering

Final holdout:
  future or recent cases not used to define graphlets

4. Graphlet Discovery Without Knowing the Right Question

The user does not know the right question yet. Therefore the first pass should not ask one fixed question.

Instead:

1. Extract typed local concepts.
2. Map them into high-dimensional role vectors.
3. Project near-edge dominant concepts.
4. Sort by culture, tradition, time, mechanism family.
5. Generate candidate graphlets from repeated local neighborhoods.
6. Rank graphlets by recurrence, independence, and anomaly pressure.
7. Ask which graphlets deserve formal testing.

This is exploratory, but not undisciplined.

5. Near-Edge Dominant Concept Projection

A "near-edge dominant concept" is a concept that repeatedly appears near high-salience edges.

Examples:

whisper
temptation
possession
confession
inversion
sacrifice
hidden hierarchy
false light
blackmail
purification
awareness
drain
devouring
initiation
boundary crossing

Algorithm:

For every concept node:
  collect incident edges
  weight edges by salience, bridge quality, and graphlet recurrence
  compute local edge-dominance score
  project concept into role-vector space
  cluster by canonical role proximity
  sort by culture and time

Formula:

edge_dominance(concept c) =
    Σ_edges incident to c [
      edge_salience
    * bridge_quality_weight
    * recurrence_weight
    * anomaly_pressure
    ] / degree_penalty(c)

Degree penalty prevents generic concepts like "evil" or "god" from dominating merely because they occur everywhere.

6. Candidate Graphlet Generator

Candidate graphlets should be generated from local neighborhoods.

For each high-edge-dominance concept:

1. Take radius-1 and radius-2 ego neighborhoods.
2. Convert local nodes to role labels:
   LocalConcept -> CanonicalRole
   Person -> ActorRole
   Institution -> InstitutionRole
   Event -> EventRole
   Practice -> RemedyRole
3. Convert edges to relation families:
   WHISPERS, TEMPTS, CORRUPTS, POSSESSES, INVERTS, RECRUITS, ISOLATES, REDEEMS, PURIFIES.
4. Canonicalize the graphlet.
5. Count recurrence across independent subgraphs.
6. Penalize source-family leakage.
7. Rank graphlet candidates.

Graphlets are not defined only by text similarity. They are defined by relation shape.

7. Graphlet Candidate Score

candidate_score(g) =
    recurrence_across_independent_subgraphs(g)
  * average_edge_quality(g)
  * average_mapping_confidence(g)
  * cross_cultural_diversity(g)
  * temporal_diversity(g)
  * anomaly_pressure(g)
  * prediction_potential(g)
  * anti_overfit_penalty(g)

Where:

prediction_potential(g)

means:

Does this graphlet imply missing edges/nodes that can be searched later?

If a graphlet cannot generate testable latent structures, it remains descriptive, not predictive.

8. Benchmark Corpus Table

Initial corpus inventory should be a CSV with these fields:

corpus_id,title,tradition,culture,language,date_start,date_end,source_type,source_family_id,split,holdout_status,notes

Example rows:

christian_nt_gospels,Canonical Gospels,Christianity,Near Eastern/Greco-Roman,Greek,0050,0100,religious_text,nt_canon,train,not_holdout,"Satan/temptation/demon motifs; remedy/prayer/discernment"
islam_quran,Quran,Islam,Arabian,Arabic,0610,0632,religious_text,quran,validation,not_holdout,"Iblis, shayatin, jinn, waswasa"
buddhist_pali_mara,Pali Canon Mara passages,Buddhism,Indian/Pali,Pali,-0400,0200,religious_text,pali_canon,validation,not_holdout,"Mara temptation and awakening obstruction"
algonquian_wetiko,Wetiko motif sources,Indigenous,Algonquian,English/various,1800,2026,anthropology_or_oral_tradition,wetiko_family,test,not_holdout,"Consumption/greed/psychic contagion"
jonestown_fbi,FBI Jonestown files,Modern cult,United States/Guyana,English,1978,1980,official_record,fbi_jonestown,final_holdout,holdout,"Coercion, political protection, murder-suicide, source suppression"

9. What to Keep Out of Sample

Keep at least these out initially:

Aum Shinrikyo
NXIVM
some Epstein/Maxwell materials
modern algorithmic radicalization
clinical intrusive-thought literature
a selected non-Western tradition not used in role definition
one unknown local corpus from user's documents

The unknown local corpus is especially useful. It functions like true out-of-sample data.

10. First Questions to Generate, Not Answer

The system should output questions like:

Which canonical roles are most edge-dominant across cultures?
Which graphlets recur across the most independent traditions?
Which graphlets recur in both religious/cultural material and modern cult/intelligence material?
Which graphlets generate the sharpest latent predictions?
Which cultures break the proposed universal pattern?
Which concepts are false friends: semantically similar but structurally different?
Which source families drive apparent convergence?
Which graphlets disappear after source-family de-duplication?
Which missing edges are predicted by multiple graphlets independently?

11. Success Criteria for the Benchmark

A strong benchmark design should produce:

1. Candidate graphlets before holdout.
2. Held-out cases not used in graphlet design.
3. Null-model comparisons.
4. Latent predictions.
5. A ranked next-best-question queue.
6. A list of failed predictions.
7. A list of source-family leakage risks.
8. A list of culturally specific non-convergences.

The benchmark succeeds even if the metaphysical hypothesis fails, because the system may still discover real mechanisms:

coercive conversion
source suppression
legitimacy laundering
attention redirection
controlled inversion
adversary archetype