Case study

Closing the Sybil and post-quantum gaps in Marmot encrypted group messaging.

Marmot is a promising architecture for decentralized E2EE messaging: MLS (RFC 9420) for forward-secure group encryption, Nostr relays as transport. It is also, by its own threat model, unable to serve regulated industries. Any actor can mint unlimited Nostr identities. Every cryptographic primitive in the critical path is quantum-vulnerable. Spam prevention is explicitly listed as not provided. This case study walks through how Disentangle's platform closes each of these gaps on the same substrate, with a minimal integration that is additive, backward-compatible, and deployable as a single protocol extension.

TargetRegulated-industry operators of E2EE messaging
IntegrationMIP-05 identity binding + PQ ciphersuite
Engagement model60 to 90 day pilot

What Marmot is, in one paragraph.

Marmot is an open specification for decentralized encrypted group messaging, layering the Messaging Layer Security protocol (RFC 9420) on top of Nostr as a transport. Its MIP series (MIP-00 through MIP-04) defines KeyPackage publication, group state management, commit ordering, and encrypted media delivery. Identity is a bare secp256k1 Nostr keypair. Transport confidentiality and ordering are outsourced to Nostr relays, which the threat model explicitly assumes are untrusted. The protocol is interesting precisely because it is stripped to its essentials. It is also, for the same reason, incomplete for enterprise deployment.

Three missing primitives in the baseline protocol.

A regulated-industry operator reviewing the Marmot spec will flag three issues immediately. None of them are design bugs; they are explicit non-goals of the baseline protocol. That is the opening.

1. No Sybil resistance

Nostr identities are free to create. An attacker can generate ten thousand npub keys in seconds, publish KeyPackages from each, and saturate any group that accepts new members. The Marmot threat model acknowledges this under T.1.2 (credential mismatch) and in the absence of any identity-weighting mechanism. For consumer use cases this is acceptable; for a regulated messaging platform, it is disqualifying.

2. No post-quantum cryptography in the critical path

The MLS ciphersuites Marmot relies on use secp256k1, Ed25519, and P-256 for signatures, and X25519 / P-256 for key agreement. All are vulnerable to Shor's algorithm. Harvest-now-decrypt-later attacks against long-lived group keys are exactly the scenario post-quantum regulatory migration deadlines are written to prevent.

3. No spam prevention, no finality, no relay reputation

The threat model explicitly lists spam prevention as not provided. Commit ordering under conflict (MIP-03) relies on self-reported timestamps with a lexicographic tiebreaker — the spec itself calls this a "practical compromise." Relays are "relied upon for message delivery" but there is no mechanism to evaluate or weight relay honesty. These are not edge-case concerns at scale.

Disentangle closes each gap with a primitive that already exists.

The platform's identity layer, consensus layer, and cryptographic library map directly onto Marmot's open problems. No component is speculative; all of the relevant code is implemented, tested, and production-grade. Each row below references the existing Disentangle crate that provides the primitive.

Marmot gap Disentangle primitive Effect
No Sybil resistance on Nostr identities disentangle-consensus topological mass + disentangle-identity coherence tiers 5:1 attacker ratio limited to ~7% of honest mass
Quantum-vulnerable MLS ciphersuites disentangle-crypto ML-DSA-87, ML-KEM-1024, SHA3-256 NIST Level 5 PQ security, FIPS 203/204 compliant
No spam prevention, no rate limit disentangle-zkp ZK-verified mass-weighted rate limiting New identities start at bucket 0; additional IDs don't add throughput
Timestamp-based commit ordering disentangle-consensus::resolve_conflict Topological-mass tiebreaker; deterministic, non-spoofable
No relay reputation Curvature-weighted relay selection Clients prefer relays with demonstrated delivery track record
Flat admin list, no delegation disentangle-identity::Capability with delegation chains Graduated permissions, revocation granularity, misuse detection
No structural audit for group compromise disentangle-membrane coherence filtering Coordinated infiltration detected as SimHash clustering

Cryptographic primitive mapping.

Replacing the classical crypto in the MLS layer is a library-level change with well-defined size implications. Signatures grow from 64 bytes to 4,627. Public keys grow from 33 bytes to 2,592. Nostr already handles variable-size events, and the MIP-04 encrypted media spec demonstrates the transport can carry large payloads; a hybrid approach (classical secp256k1 at the Nostr event layer, PQ at the MLS layer) preserves relay compatibility while adding PQ protection where it matters most.

Signatures

secp256k1 → ML-DSA-87

secp256k1 · 64 B sig · 33 B pk · classical
ML-DSA-87 · 4,627 B sig · 2,592 B pk · FIPS 204 L5
Key encapsulation

X25519 → ML-KEM-1024

X25519 · 32 B ct · classical ECDH
ML-KEM-1024 · 1,568 B ct · FIPS 203 L5
Hash

SHA-256 → SHA3-256

SHA-256 · Merkle-Damgard
SHA3-256 · Keccak · PQ collision-resistant
Transport

TLS 1.3 → Noise-XX + ML-KEM-1024

TLS 1.3 · classical
Noise-XX + ML-KEM-1024 re-keying

A single MIP, a single new field, zero breaking changes.

The highest-leverage integration is identity weighting via ZK-verified coherence tiers. It closes the most fundamental threat-model weakness (Sybil resistance) with one additive field on the existing marmot_group_data extension (0xF2EE). The integration is additive — clients that do not support the new field ignore it and operate as today — and the cryptographic verification is a single 12-millisecond function call per KeyPackage.

Extension to marmot_group_data, version 3. Two new fields, constant size. min_coherence_tier (1 byte) and mass_attestation_root (32 bytes, a Merkle root of per-member ZK reputation proofs). Proofs are distributed out-of-band as kind-447 Nostr events, the same pattern MLS uses for the ratchet tree. GroupContext size increase: 33 bytes regardless of group membership count.

Groups that set min_coherence_tier >= Participant automatically exclude all identities that have not accumulated meaningful topological mass through genuine participation. This closes the Sybil resistance gap with a single field.

Four phases from zero-risk observation to full PQ substrate.

01

Shadow mode

Add ReputationVerifier to the Marmot client. Accept ReputationClaim proofs attached to KeyPackages. No enforcement — just log trust levels alongside existing behavior. Zero risk, zero user impact. Operators get a telemetry feed showing who would be admitted and who would be rejected under each potential threshold.

02

Opt-in Sybil gate

Add min_coherence_tier to marmot_group_data version 3. Groups opt into Sybil resistance by setting the field. Backward compatible: clients without v3 support ignore the field entirely. Regulated-industry groups can require Participant-tier or higher membership; consumer groups can leave it at Observer.

03

Mass-weighted rate limiting

Enforce rate limits on kind 445 (group messages) using ZK-verified mass. New identities start at bucket 0 — creating more identities does not multiply throughput. Relay operators can enforce client-side or server-side. Spam prevention gap closed without breaking metadata privacy (the verifier learns "mass above threshold," not the sender's identity or exact mass).

04

PQ ciphersuite and DAG-based commit ordering

Define a new MLS ciphersuite combining ML-DSA-87 signatures, ML-KEM-1024 key encapsulation, and SHA3-256. Replace the MIP-03 timestamp tiebreaker with topological-mass conflict resolution. At this point Marmot is running on the Disentangle substrate end-to-end: every MIP event is a DAG transaction, mass accumulates through normal messaging, and the Sybil-resistance gate activates organically as the group matures.

A regulated-industry E2EE messenger, on open standards.

The Disentangle-plus-Marmot stack is a credible answer to a procurement question that currently has none: is there a decentralized, standards-based, post-quantum, Sybil-resistant, audit-friendly E2EE messenger for finance, legal, health, and defense? Signal is centralized. Matrix has no PQ story and no Sybil resistance. Proprietary enterprise tools (Symphony, Wickr, MS Teams) are centralized with classical crypto. The Marmot + Disentangle integration sits at the only open-standards, decentralized, PQ-native, Sybil-resistant point in the design space.

For an operator, the commercial pitch writes itself: deploy an MLS-based group messenger on Nostr relays, extend the KeyPackage with a Disentangle mass attestation, run the reference cluster on Oracle Always Free, and sell per-seat licensing to regulated buyers who cannot buy anything else. Disentangle licenses the patent-pending IP; the operator ships the product.

What we are not hiding.

  • Key sizes. ML-DSA-87 signatures are 72× larger than secp256k1. KeyPackage events grow from ~500 bytes to ~8 KB. Relay bandwidth and storage costs go up proportionally. Nostr already handles variable-size events, but this is a real operational consideration.
  • Bootstrap window. A new identity starts with zero topological mass. The bootstrap ramp means curvature throttling becomes fully effective only after the group's DAG reaches ~6,000 depth — roughly three to twelve days of normal group activity. During bootstrap, min_coherence_tier should remain at Observer. This is the correct default: new groups are not the infiltration target.
  • Dependency surface. Minimal integration adds ~50 crates and ~5 MB of binary size (primarily pqcrypto-* and a plonky3 verifier subset). Full integration adds ~200 crates and ~15 MB. A Marmot client already depending on an MLS library and a Nostr client roughly doubles its dependency count for minimal integration.
  • Nostr compatibility. Replacing secp256k1 at the Nostr event layer breaks NIP-01 compatibility. The hybrid approach (classical secp256k1 on Nostr, PQ on MLS) is the recommended path — a DID document can hold both verification methods, which is standard DID practice.

None of these are blockers. All are well-characterized in the full technical integration analysis and none require primitive research — this is an engineering integration, not an R&D project.

Want to walk through the integration?

We can share the full 500-plus-line technical integration analysis under NDA, walk through the API boundary in detail, and scope a 60 to 90 day pilot against a concrete group-messaging deployment you control.