another-im/interop
Watch
9
0
Fork
You've already forked interop
 0
Code Issues 5 Pull requests 1 Releases 7 Activity Actions

Batch 7: cover the untested features, code coverage, hostile-server fuzzing #29

New issue
Open
opened 2026-11-10 21:01:11 +00:00 by asxpi · 0 comments
asxpi commented 2026-11-10 21:01:11 +00:00
Member
Copy link

Batch 7: cover what the harness has never run, measure how much of libtacky it reaches, and attack it with hostile server input. Starts after batch 6 (#25) is merged.

At the start, 98 of the 208 rows of libtacky's docs/features.md show no run in FEATURES.md, most because they were ai_ok = false (agents' runs of those were hidden); 11 have no scenario at all. Only 3 cells were run by a person.

Every finding of batch 7, fuzzer included, is recorded in this issue, not in #1 or #9.

Feature list per spec (scenarios)

Written 2026-11-11 from libtacky develop 65a31465 (docs/features.md, HANDBOOK.md, docs/protocol/), interop main (features/*.toml, tools/ai/scenarios/, runs/, AI-RESULTS.md, tools/ai/NIGHT-2026-11-05.md, vms/AGENTS.md) and #1, #9, #24, #25. One bullet with an id (0166-2) is one scenario, which is one run per server. Each scenario holds the main path, the MUST edge cases and at least one malformed or unauthorised case, as 7a asks.

Names used below: T = tackyd-json under test (ai{N}a); S = slixmpp (ai{N}b); T2 = a second tackyd as the peer (ai{N}b, a store of its own); T' = a second tackyd device of T's own account; own = a slixmpp session of T's account; D1, D2 = slixmpp-omemo devices made for the run; fake = a local server on 127.0.0.1 as in XEP-0215.py; pr / ej / ai = Prosody, ejabberd, anotherim-server.

Section numbers are quoted from the existing RULES lists or from libtacky's source. Where a line gives none, the author takes it from the spec (step 0 of tools/ai/PROMPT.md), never from memory.

What the earlier batches already did (read before writing anything)

  • "No run" in FEATURES.md does not mean "never run". FEATURES.md leaves out low-confidence runs (🔸, ai_ok = false). Of the rows 7a lists, only 11 have no scenario and no run: XEP-0068, 0176, 0260, 0261, 0293, 0294, 0320, 0338, 0339, 0343, 0453. Every other row has a scenario in tools/ai/scenarios/, a RULES list, filled steps in features/<slug>.toml and runs on all three servers. Do not write those again: rerun them on the batch's build and add what is listed here.
  • Many of those runs are on v1.2.0-rc2, not on develop. "rc2" below marks them; they are rerun unchanged first, so a change since rc2 shows as a difference.
  • Headless tackyd-json has no native media backend. media status of the interop build answers backend "" ("neither webrtc nor datachannel is in this build"), so calls start never reaches a session-initiate; that is why XEP-0166 part C, XEP-0298 and XEP-0507 ended as INFO or na. The remote backend opens in the same binary (--media-backend remote: audioCalls, videoCalls, dataChannel, renegotiate, trickleIce; checked on tackyd-json-develop-65a31465). With it the scenario plays the page: it answers calls <MediaOpen> with a canned SDP (calls local_description, local_candidate, media_state) and reads calls <MediaRemote> / <MediaCandidate>. All Jingle call scenarios below use it.
  • spans:0, live-location:0 and service:0 are read-only in libtacky (rows: "Not sent", "read"). Run libtacky against libtacky and nothing crosses. For these three S stays the sender; the libtacky-against-libtacky part is the second device and the archive.
  • Gajim 2.6 has no Jingle file transfer (clients.toml lacks: XEP-0234, 0260, 0261). 7d names Dino and Gajim for it; only Dino and anotherim-web can run those cells.

Rules for every scenario (mistakes made before)

  • Server behaviour is INFO, never libtacky's fail. Known differences are in tools/ai/differences.toml: ejabberd's mod_caps and Prosody's mod_pep send their own disco queries (count only the client's full JID); Prosody delivers each PEP notification twice; anotherim-server archives a message to a blocked JID before bouncing it. A fail that is the server's gets --cause server (XEP-0065 on Prosody was relabelled for this).
  • A behaviour libtacky's row documents is not a bug (C12, XEP-0153 photo hash before the vCard). Quote the row in the notes instead.
  • The store outlives the run. Make every body, node, room, feature name and device unique to the run; read and restore every setting touched (OMEMO switch and mode, file_transfer_p2p, attachment_autofetch, transports, auto_trust, blind trust). Take run-made OMEMO devices off the device lists and delete their bundles. RFC 7622's junk chats and XEP-0478's extra device stayed in the stores for days.
  • iq ids restart with each tackyd session. Scope every lookup with since=mark().
  • Tacky.send_raw goes through the debug tap and is counted by stream management (XEP-0198 on ejabberd resent it). Do not use it around a resume or a stream error.
  • Discover, do not hard-code: MUC, pubsub, upload and proxy services by disco. Print what depends only on the server as INFO.
  • Wait long enough for big stanzas: ejabberd delivered an 88 KB stanza after about 15 s.
  • Shell and disk: tools/ai/aish, never nix develop .#ai (it filled the disk twice). Scratch in mktemp -d; delete export folders that hold history.
  • Secrets: never print or store passwords, TURN credentials, FAST tokens, invite tokens, registration passwords or an account link_device URI (it holds osec). Check the base64 forms too. Room passwords from config forms appeared in clear in peer-slixmpp.log once.
  • slixmpp 1.17: no Jingle (raw IQs), xep_0202 cannot answer time queries, plugins through p.xmpp.plugin[...]. OMEMO 2 is python-omemo (slixmpp-omemo).
  • A run the server restart interrupted is finished na and redone. A run a revised scenario replaced is finished na with "superseded by ".
  • Low confidence wording: notes begin low confidence: checked …; not checked: ….

Harness changes these scenarios need (before 7a starts)

# Change Needed by
H1 Tacky(..., media_backend="remote") (today --media-backend auto is fixed in tacky.py) and a Page helper in common.py: canned SDP offers and answers (audio, audio+video, data channel), candidates, media_state, a byte pipe for <MediaData> / data_message. Fixtures: libtacky's tests/client/test_calls_remote.tcl (CHROME_OFFER, CHROME_CANDIDATE). every call and data-channel scenario
H2 A second tackyd in one run: its own log_name (peer-tackyd.log), events log (client-events.log is fixed today, so two instances would share it) and debug log; interop start --under-test tackyd --client tackyd for a kind = "peer" feature. every T×T2 scenario
H3 A second store for the same account (Tacky(state_dir=…)), for T'. sealed:1, omemo-link:0, XEP-0450
H4 also = [...] in run.toml (already in 7a). A run that fails one also spec's rules is rerun split, so one spec's bug does not fail the others' rows. call and OMEMO runs
H5 A shared fake server (tools/ai/fake.py: direct TLS on 127.0.0.1, pinned certificate, PLAIN, programmable disco and answers), from XEP-0215.py and RFC-5929.py, which each carry a copy. XEP-0048, 0068, 0452, 0488, storage:0

Open points for the maintainer

Decided 2026-11-11:

  1. Every row may count: ai_ok = true in every features/*.toml. VERIFIED stays a person's pass; rows with a display or media part are still run confidence = "low".
  2. XEP-0452 on the real prosody.asxp.io, no fake server: the maintainer enables mod_muc_mention_notifications there (on the MUC component: modules_enabled = { "muc_mention_notifications" } and muc_mmn_notify_unaffiliated_users = true). 0452-2's fake part is dropped; its hostile cases go into 0452-1 on Prosody. ejabberd and anotherim-server stay [na].
  3. urn:tacky:storage:0: the fake server (tools/ai/fake.py) is its only run.
  4. XEP-0453: the one offline check 0453-1, recorded on Prosody; ejabberd and anotherim-server have an [na] reason.
  5. Run count: 90 scenario ids (P2 has 31 with 0364-2), about 260 runs, about 170 without P2.

Harness H1–H5 and these decisions are in the batch 7 harness PR (branch ai/batch-7-harness).

P0

XEP-0166 Jingle · has XEP-0166.py (R1–R17, file session over IBB), develop: pr ✓ ej ✓ ai ✓ (C31 fixed)

  • 0166-1 T×S, rerun, then add to the same file session: content-modify (answered, senders unchanged), content-remove (ends the session), description-info and security-info (unsupported-info), a responder's transport-replace crossing T's own (<conflict/> + <tie-break/>). R16 (a refused sid sent again) becomes a check, not INFO. Hostile: initiator naming a third JID; a session-initiate from a bare JID; a sid of 4 KB with markup characters; a 9th open incoming session (XEP-0234 row: resource-constraint).
  • 0166-2 T(remote)×S, raw Jingle, call session (new). T calls start, S answers the XEP-0353 proceed, the page gives the offer: session-initiate has initiator, every <content/> one description and one transport. S session-accept → <MediaRemote type answer>, media_state connected → <Active>. Then: S's transport-replace (refused; record the condition), S's content-add video → <MediaRemote offer>, the page's answer → content-accept, <Renegotiated>; S's content-remove; T's own content-add (a mid-call local_description offer), S content-reject → <Renegotiated rejected>; crossing session-initiates (tie-break); session-terminate both ways, then unknown-session. Hostile: session-accept from S's other resource; a second session-accept; media_failed from the page (session-terminate with a reason, <Failed>); a propose while a call is up (busy, <Ended> missed).
  • 0166-3 T(remote)×T2(remote), both ways (new): audio call, video call, reject, hang up from each side, a call to an offline peer. Pass when every action is acked, both sides end in the same state, and the SDP that reaches the far page has every line class the near page sent (codecs, rtcp-fb, extmap, ssrc, group, fingerprint, candidates). also = [XEP-0167, XEP-0353].

XEP-0176 Jingle ICE-UDP · no scenario

  • 0176-1 T(remote)×S (new). T's initiate carries ufrag and pwd from the page's SDP; every trickled transport-info carries both (libtacky cites XEP-0176 §5.3) and a full candidate (component, foundation, generation, id, ip, port, priority, protocol, type; rel-addr and rel-port for srflx and relay). S's candidates sent before the page opened are buffered and replayed as <MediaCandidate> after <MediaRemote>. A candidate under the known ufrag without a pwd is applied. New ufrag and pwd together are an ICE restart: on remote (iceRestart false) record what T does as INFO. TCP candidates are not signalled: S's protocol='tcp' candidate and the page's tcp candidate cross neither way. Hostile: priority 2^32, port 70000, an ip that is none, a ufrag with spaces or 300 characters, a candidate for an unknown content, 500 candidates at once, a transport-info from another resource. Pass when each is refused or dropped, no <MediaCandidate> carries it, and the session lives.
  • 0176-2 T(remote)×T2(remote) (new): trickle both ways with scripted host, srflx and relay candidates; <MediaOpen>'s ice_servers equal extdisco fetch on that server (stun and turn on all three since #13). also = [XEP-0215].

XEP-0320 DTLS-SRTP in Jingle · no scenario

  • 0320-1 T(remote)×S (new). T's initiate has <fingerprint xmlns='urn:xmpp:jingle:apps:dtls:0' hash setup/> in every transport, equal to the page's a=fingerprint and a=setup; S's accept with setup='active' reaches the page so; as responder the page's answer role is in T's session-accept. calls <Security>: dtls_local, dtls_remote, 7 SAS indices, encryption none. Hostile: an accept with no fingerprint; a fingerprint that is not colon-separated hex; setup='bogus'; two contents with different fingerprints; a fingerprint changed by a later transport-info. Pass when the malformed ones never reach the page and the call ends with a reason, not <Active>.
  • 0320-2 T(remote)×T2(remote), OMEMO on, both devices trusted and advertising urn:tacky:jingle:dtls-omemo:0 (new): the fingerprints cross sealed as docs/protocol/jingle-dtls-omemo-0.md has it, <Encryption> / <Security> say omemo with the device and its trust, both sides show the same SAS. Then T2's device undecided: the trust says so. Hostile (S in a third session): S replays T2's sealed element in a session of its own → <Failed>, session-terminate security-error. also = [urn:tacky:jingle:dtls-omemo:0].

XEP-0420 Stanza Content Encryption · has XEP-0420.py (R1–R19), develop: pr ✓ ej ✓ ai ✓ (S2 fixed)

  • 0420-1 T×S+D1, rerun, then add what 1.2.0–1.2.4 brought: a retraction in an OMEMO 2 chat is sealed (<retract/>, fallback body and <fallback/> inside, no body beside), a bare <retract/> beside the ciphertext, none with retract_hint 0; a <time/> more than 30 days before a stamp the server vouches for (D1 sends while T is offline, T catches up) is refused <DecryptFailed> sce; an OMEMO 2 room message without <to/>, or with another room's, is refused. R19 stays INFO for receipts, markers and reactions (no server advertises urn:tacky:push:sealed-signals:0, #9 S15).
  • 0420-2 T×T2, OMEMO 2 both (new): body, reply, correction, retraction, reaction, file share, in a 1:1 chat and in a members-only non-anonymous room. Pass when no <DecryptFailed> appears and R6–R8 hold on every stanza of both logs. A helper check_sce(stanza) in common.py runs the same rules in every OMEMO 2 run (XEP-0384, 0434, 0450, 0454, jet-omemo:2), which then carry also = [XEP-0420].

P1, protocol

XEP-0048 Bookmarks · has XEP-0048.py (R1–R10), rc2: ✓✓✓, R9–R10 never ran (all three servers have PEP bookmarks)

  • 0048-1 T×S+own, rerun.
  • 0048-2 T×fake without PEP bookmarks (new, H5): R9, R10: private XML fetched on connect, the whole <storage/> written back with other conferences, <url/> and foreign children unchanged. Hostile: 5000 conferences, a malformed jid, autojoin='maybe', the same jid twice.

XEP-0050 Ad-Hoc Commands · has XEP-0050.py (R1–R15), develop: ✓✓✓ (M2 fixed)

  • 0050-1 T×S, rerun, add: a required field left empty is refused before any IQ (row: validated before submit); the default action when <actions/> names no execute; the responder's bad-action, bad-sessionid and session-expired reach the caller. Hostile: the sessionid changed mid-session; a result from another JID under the same iq id; a 500-field form; an unknown note type.

XEP-0060 Publish-Subscribe · has XEP-0060.py (R1–R16), repro-C11.py, develop: ✓✓✓

  • 0060-1 T×S, rerun the full scenario on all three (ej and pr last ran only the C11 repro), add: subscriptions survive a restart of T; a pending or unconfigured subscribe answer is reported; a notification from the right service with a node T never subscribed to, and from another service with the right node, is no pubsub <Notification>. Hostile: an event with 1000 items; a 200 KB payload; <delete><redirect/></delete>. Not checked (row): §6.3, §8.7, §8.8, collections.

XEP-0066 Out of Band Data · has XEP-0066.py (R1–R8), rc2: ✓✓✓

  • 0066-1 T×S, rerun, add the 1.2.2 and 1.2.5 rules: a room occupant's picture is not fetched by itself; contacts means subscription both (a from-only contact's file waits); a stored everyone reads back as contacts. Hostile: a url with userinfo, of 4 KB, aesgcm:// with a broken fragment, https to a local listener (count connections).

XEP-0068 Field Standardization · no scenario

  • 0068-1 T×S+fake (new). Out: every x:data submit T sends in the run carries the right FORM_TYPE: MAM query, push enable, pubsub node config and publish-options, MUC config and muc#register, a CAPTCHA answer, an ad-hoc stage (the responder's own). In: on the fake, an upload form whose FORM_TYPE is missing, not hidden, or another value gives no max-file-size (storage-0 §5); a CAPTCHA with a wrong FORM_TYPE is not told (XEP-0158 R2); a caps answer with two forms of one FORM_TYPE is not cached (XEP-0232 R7).

XEP-0082 Date and Time Profiles · has XEP-0082.py (R1–R8), rc2: ✓✓✓

  • 0082-1 T×S+own, rerun, add malformed forms: 23:59:60, year 0000 and 10000, lower-case t and z.

XEP-0203 Delayed Delivery · WIP · has XEP-0203.py (R1–R10), rc2: ✓✓✓

  • 0203-1 T×S+own, rerun, add: after a XEP-0198 resume a resent stanza's server stamp is taken, never earlier than that session's start; a forged from=DOMAIN stamp 16 s after login with no inbound traffic gets the arrival time (the window closed). o3 stays INFO (the row's open issue).

XEP-0221 Data Forms Media Element · has XEP-0221.py (R1–R11), develop: ai ✓; pr ✗ and ej ✗ are rc2 runs (S5, fixed in 1.2.1)

  • 0221-1 T×S, rerun on pr and ej first: their cells still show the rc2 fail. Add: a sha256+ cid; 100 <uri/>s in one <media/>; width 65536.

XEP-0231 Bits of Binary · has XEP-0231.py (R1–R12), rc2: ✓✓✓

  • 0231-1 T×S, rerun, add: a cid with max-age 2 is fetched again after 3 s; a result for T's BoB get from another JID is dropped; a fetched blob over the 64 KiB cap is refused; no fetch from a non-contact.

XEP-0260 Jingle SOCKS5 Bytestreams Transport · no scenario (parts ran inside XEP-0065.py and XEP-0234.py part S)

  • 0260-1 T×S, S with the SOCKS5 client and server of XEP-0065.py (new). Candidates carry cid, host, jid, port, priority, type; dstaddr = SHA1(sid + initiator + responder); candidate-used and candidate-error both ways; both sides used: the higher priority wins, a tie goes to the initiator's; a proxy is activated by its offerer (<activated/>), <proxy-error/> moves to IBB by transport-replace; mode='udp' → unsupported-transports; file <Update> says transport socks5. Hostile: 12 candidates (trap listeners count: 8 dialled at most); candidates on 0.0.0.0, 224.0.0.1, 255.255.255.255, 127.0.0.1 (none dialled); candidate-used naming an unknown cid; <activated/> for a cid nobody used.
  • 0260-2 T×T2 (new): a 1 MiB file each way with transports [socks5], direct, then through the server's proxy where it is reachable (ejabberd's proxy65 announces 127.0.0.2, #9 P6: INFO there). Pass when the bytes are intact and verified.

XEP-0261 Jingle IBB Transport · no scenario (parts ran inside XEP-0047.py)

  • 0261-1 T×S (new). The responder's smaller block-size is honoured; the transport's sid is the <open/>'s; after both SOCKS5 sides report candidate-error T sends transport-replace to IBB and continues on S's transport-accept; as responder T accepts S's transport-replace to IBB. Hostile: transport-replace to an unknown namespace (transport-reject); block-size 0, 70000 and abc in the accept; a transport-accept nobody asked for (out-of-order). Not checked: the responder's replace when the account's proxy takes the data channel away.
  • 0261-2 T×T2 (new): transports [ibb], 1 MiB each way, intact, transport ibb.

XEP-0264 Thumbnails · has XEP-0264.py (R1–R10), repro-C15.py, develop: ✓✓✓ (ej and pr by the repro only)

  • 0264-1 T×S, rerun the full scenario on all three, add 1.2.5: an upload's SFS share names the thumbnail twice, by cid and as a data: URI when it is at most 16 KiB; between 16 and 64 KiB by cid only. The same inside an OMEMO 2 envelope is checked in 0420-2.

XEP-0292 vCard4 Over XMPP · has XEP-0292.py (R1–R11), repro-C16.py, develop: ✓✓✓ (ej and pr by the repro only)

  • 0292-1 T×S, rerun the full scenario, add: org, title, role, impp, tz, pronouns and adr round trip; two vcard set back to back on Prosody leave the second on the node (the retried publish that overwrote a newer one, NIGHT 2026-11-05); no echo of the server's converted item. Check for #9 C47: after vcard set, S reads T's vcard-temp: R10 says FN, N and NICKNAME are mirrored, the VM runs saw Kaidan read an empty card on ej and ai. Report which holds. Hostile: a 200 KB item; an item id that is not current; two items; a forged event from a full JID.

XEP-0293 RTP Feedback, XEP-0294 RTP Header Extensions, XEP-0338 Jingle Grouping, XEP-0339 Source-Specific Media · no scenario

  • 0293-1 T(remote)×S, one run with also = [XEP-0294, XEP-0338, XEP-0339] (new). One SDP fixture, both directions (the page's SDP → T's Jingle; S's Jingle → <MediaRemote> SDP), a call T starts and a call S starts:
    • 0293: a=rtcp-fb:PT nack pli, ccm fir, transport-cc, a=rtcp-fb:* … and trr-int ↔ <rtcp-fb type subtype/> and <rtcp-fb-trr-int value/> in urn:xmpp:jingle:apps:rtp:rtcp-fb:0, under the payload-type or, for *, the description. Hostile: an <rtcp-fb/> without type; a non-numeric trr-int; 300 of them.
    • 0294: a=extmap:ID[/direction] URI [params] ↔ <rtp-hdrext id uri senders/> in urn:xmpp:jingle:apps:rtp:rtp-hdrext:0, the direction as senders for an initiator and for a responder, parameters kept, a=extmap-allow-mixed ↔ <extmap-allow-mixed/>. Ids: 1, 14, 15, 255, 4096 accepted; 0, 256, 4352, -1 and x refused (jinglesdp.tcl).
    • 0338: every a=group (BUNDLE and LS) ↔ <group semantics><content name/></group> in urn:xmpp:jingle:apps:grouping:0. A group naming a content the session lacks loses that name; semantics that is no token is refused.
    • 0339: a=ssrc:ID name:value, a valueless parameter and a=ssrc-group:FID a b ↔ <source ssrc><parameter/></source> and <ssrc-group semantics> in urn:xmpp:jingle:apps:rtp:ssma:0. ssrc 4294967295 accepted; 4294967296, -1 and abc refused.
    • Each spec's disco feature is in T's disco#info (INFO where the spec asks for none). Pass when every mapping is exact both ways and each refused input ends in an IQ error or a session-terminate with a reason, never a crash or a half-applied description.

XEP-0300 Hash Functions · WIP · has XEP-0300.py (R1–R15), rc2: ✓✓✓

  • 0300-1 T×S, rerun, add: a sha-256 value of 31 bytes; algo='SHA-256'. Not checked (row): BLAKE2b.

XEP-0317 Hats · has XEP-0317.py (R1–R10), rc2: ej ✓ ai ✓ pr na (Prosody 13 runs no hats commands)

  • 0317-1 T×S, rerun, add: <hats/> in a message is not read; a 10 KB title. Prosody: R1 and R10 only, verdict na as before.

XEP-0334 Message Processing Hints · has XEP-0334.py (R1–R11), rc2: ✓✓✓

  • 0334-1 T×S+D1, rerun, add: a sealed retraction (1.2.4) carries <store/>; hints on a carbon and on an archived copy never drop the row.

XEP-0343 WebRTC Data Channels in Jingle · no scenario

  • 0343-1 T(remote)×S (new). Transports [datachannel, ibb]: T's offer has the urn:xmpp:jingle:transports:webrtc-datachannel:1 transport around ice-udp with sctp-port and max-message-size; <MediaOpen data=1 role offer>; after data_state open the file leaves as <MediaData> chunks and <MediaDataClose>; the bytes equal the file. S offers: file accept, the page feeds data_message chunks, the file is kept verified. media_state failed → transport-replace to the next transport (XEP-0260, 0261 rows). Hostile: more bytes than <size/>; data_message before open; a wrong hash (not kept, <media-error/>); a non-numeric sctp-port (5000 assumed).
  • 0343-2 T(remote)×T2(remote), the scenario pipes the bytes between the pages (new): a file each way in a clear chat, then in an OMEMO 2 chat (JET-OMEMO 2, fingerprints sealed). also = [XEP-0391, urn:tacky:jingle:jet-omemo:2, urn:tacky:jingle:dtls-omemo:0].

XEP-0367 Message Attaching · has XEP-0367.py (R1–R15), develop: ✓✓✓ (C6 fixed)

  • 0367-1 T×S+own, rerun, add: an attach-to naming its own message; A → B → A; 100 captions for one file; in a room, the same nick under another occupant id is not folded.

XEP-0372 References · has XEP-0372.py (R1–R10), rc2: ✓✓✓

  • 0372-1 T×S, rerun, add: 1000 references in one message; a mention after XML character references in the body.

XEP-0380 Explicit Message Encryption · has XEP-0380.py (R1–R13), repro-C17.py, develop: ✓✓✓ (ej and pr by the repro only)

  • 0380-1 T×S+D1, rerun the full scenario, add #9 M27: the stored record of an OMEMO 2 message T sent (raw_xml) names urn:xmpp:omemo:2. Hostile: 50 <encryption/> elements; a 5 KB namespace; a name with markup and bidi overrides.

XEP-0385 SIMS · has XEP-0385.py (R1–R18), repro-C13.py, develop: ✓✓✓ (ej and pr by the repro only)

  • 0385-1 T×S, rerun the full scenario, add: 30 sources; a size of -1 and of 2^64; a name with a path, a bidi override and a NUL. legacy_file_metadata 0 (#1 S3) stays in the XEP-0384 runs, which get also = [XEP-0385].

XEP-0391 Jingle Encrypted Transports · has XEP-0391.py (R1–R17), rc2: ✓✓✓

  • 0391-1 T×S+D1, rerun, add 1.2.5 (#9 P4): Dino's cipher spelling is sent and both spellings are read; an offer of 256 MiB + 1 is refused before any byte. The libtacky-against-libtacky transfer is 0343-2.

XEP-0398 Avatar to vCard Conversion · has XEP-0398.py (R1–R6), rc2: ✓✓✓

  • 0398-1 T×S, rerun, add: a vCard PHOTO of 5 MB, of type SVG, and an EXTVAL http url (nothing fetched); the conversion feature in a forged disco#info result is not believed.

XEP-0428 Fallback Indication · has XEP-0428.py (R1–R10), develop: ✓✓✓ (C4 fixed)

  • 0428-1 T×S, rerun, add: 500 <fallback/> elements; start = end; overlapping and unsorted ranges (row: merged).

XEP-0434 Trust Messages · WIP · has XEP-0434.py (R1–R13), rc2: ✓✓✓

  • 0434-1 T×S+D1+D2, rerun, add: two <trust-message/> in one message (R1: exactly one); a usage other than ATM; a trust message in the clear; a contact's device naming a third JID as key owner (ignored). Not checked (row): OMEMO 0.3 keys, the archive.

XEP-0446 File Metadata · has XEP-0446.py (R1–R12), rc2: ✓✓✓

  • 0446-1 T×S, rerun, add: a <size/> that is not the bytes'; a negative <length/>; width 0 and 10^9; a 4 KB name; two <file/> elements.

XEP-0450 Automatic Trust Management · WIP · has XEP-0450.py (R1–R13, R6 not run), rc2: ✓✓✓

  • 0450-1 T×S+D1–D4, rerun, add R6 (a key not fetched yet) and the bounds: the 33rd trust message from a device not authenticated and the 257th unknown key are not kept.
  • 0450-2 T+T'×T2 (new, H3): T verifies T': T' gets every key T verified and refused; T's refusal of a T2 key goes to T' and not to T2; T2, verified, gets T's and T''s keys. Pass when the trust lists of T and T' are equal at the end.

XEP-0452 MUC Mention Notifications · has XEP-0452.py (R1–R5), rc2: na ×3 (see open point 2)

  • 0452-1 T×S, rerun on Prosody only if the module is enabled; else [na] on all three.
  • 0452-2 T×fake (new, H5): R1, R3, R5 with the room's bare JID as the sender. Hostile: a forwarded message of another room; a non-groupchat one; a stamp in the future (never later than now); a 5000-character text (cut at 1024); the same mention twice (told once); a room T does not know.

XEP-0469 Bookmark Pinning · has XEP-0469.py (R1–R7), rc2: ✓✓✓

  • 0469-1 T×own, rerun, add: own changes the bookmark's name, then T pins: the pin's republish keeps the new name; <pinned/> twice; <pinned/> with children.

urn:xmpp:mucsub:0 ejabberd MUC/Sub · has urn-xmpp-mucsub-0.py (R1–R13), develop: ej ✓ ai ✓, pr [na]

  • mucsub-1 T×S, rerun, add: a kick, a ban and the room's destruction each end the stored subscription; S unsubscribes T while T is offline, and T's list is reconciled at connect; a reaction, a correction and a retraction through MUC/Sub land on the room chat's rows; the archive catch-up files the room's copy once. Hostile: a wrapper from the MUC service's domain; a wrapper from room A carrying a message from='roomB/nick'.

urn:fluux:poll:0 ProcessOne polls · has urn-fluux-poll-0.py (R1–R27), develop: pr ✓ ej ✓, ai rc2 ✓

  • poll-1 T×S, rerun (ai first), add the room part: the tally by occupant id, a late joiner reading poll, votes and close from the archive ({urn:xmpp:mam:2}ids), <poll-results/> asked of the creator. Hostile: a results IQ from a non-occupant; 21 options; the same emoji twice; a deadline in the past; a close from the creator's nick under another occupant id.
  • poll-2 T×T2 in a room (new): poll, votes, close and results between two libtackys. Pass when both show the same tally.

P1, tacky-only

These catch crashes and regressions, not misread specs: the spec is docs/protocol/<name>.md of the tested commit.

urn:tacky:sealed:1 · has urn-tacky-sealed-1.py (R1–R23), develop: pr ✗ ej ✗ (#1 M18 left: a replayed draft is dropped without a debug line), ai rc2 ✓

  • sealed-1 T×own, rerun on all three; say for M18 whether the item is refused (the security rule) apart from whether the line is logged.
  • sealed-2 T+T' (new, H3): T' publishes its seal key and T trusts it: T seals the drafts, pins, tags and mute list again to T' under their own stamps, and T' reads them; a draft T' writes reaches T; T' set untrusted gets no key in the next item. Hostile (own): a seal key item signed by another identity; a key under a device id not on the OMEMO list.

urn:tacky:omemo-link:0 · has urn-tacky-omemo-link-0.py (R1–R15, the showing side only), rc2: ✓✓✓

  • link-1 T×own, rerun.
  • link-2 T+T' (new, H3): T account link_device, T' (an empty store) account add_from_uri: omemo <LinkVerified> on both, each trusts the other's device. A link whose ofp is changed → <LinkVerifyFailed> on T'. The same link used by a third store verifies nothing (spent). The URI is a secret: never in a log, a note or the scenario's output.

urn:tacky:storage:0 · has urn-tacky-storage-0.py (R1–R12), rc2: na ×3 (see open point 3)

  • storage-1 T×own, rerun; [na] stays while no server offers the service.
  • storage-2 T×fake with an upload and a storage item (new, H5): R2, R6–R9, R12: -personal 1 asks the storage service; file keep runs the keep command, an aesgcm:// URL goes without its fragment and comes back with it. Hostile: keep answers an http url or another host's; a stage that never completes; two storage services (the first is used).

urn:tacky:service:0 · read-only · has urn-tacky-service-0.py (R1–R15), develop: ✓✓✓ (C5 fixed)

  • service-1 T×S+own, rerun, add: the same rows after an archive catch-up (T offline while S sends); <service/> twice in one message; a 300-character kind; 200 attributes.

urn:tacky:spans:0 · read-only · has urn-tacky-spans-0.py (R1–R17), repro-S6.py, develop: pr ✓ ej ✓, ai rc2 ✓

  • spans-1 T×S+D1, rerun the full scenario on all three, add: spans inside an OMEMO 2 envelope read the same; spans beside a ciphertext are ignored (XEP-0420 R16).

urn:tacky:live-location:0 · read-only · has urn-tacky-live-location-0.py (R1–R11), develop: ej ✓, pr and ai rc2 ✓

  • live-1 T×S+own, rerun, add: moves that arrive from the archive after T was offline leave one row at the last place with no revision; a live location in a room is moved only by the same occupant id; until already past at the start.

urn:tacky:push-hint:0 · has urn-tacky-push-hint-0.py (R1–R9, in the clear only), rc2: ✓✓✓

  • hint-1 T×S, rerun.
  • hint-2 T×T2, OMEMO 2 (new): an album with a caption: <no-push kind='sources'/> and kind='caption' stand beside the ciphertext; a caption that mentions someone or answers a message carries none. Not checked: what a push service does with it (no app server in the lab).

P2, if time allows

One scenario each; "rerun" means the existing scenario unchanged on the batch's build, with the adds in the same run.

Spec Has, last result Scenario Add
XEP-0047 IBB XEP-0047.py, develop ✓✓✓ (C2 fixed) 0047-1 T×S rerun the seq wrap 65535 → 0 (block-size 16, about 1.05 MB; time box 10 min, else not checked); either side closing mid-stream
XEP-0055 Jabber Search XEP-0055.py, rc2 ✓✓✓ 0055-1 T×S rerun muc search (1.2.0) against a room S plays
XEP-0065 SOCKS5 Bytestreams XEP-0065.py, rc2 ✓✓✓ (pr relabelled: mod_proxy65) 0065-1 T×S rerun nothing new; 0260-1 covers the transport; ej's proxy announces 127.0.0.2 (#9 P6)
XEP-0080 User Location XEP-0080.py, repro-S8.py, develop ✓✓✓ 0080-1 T×S rerun the full scenario none
XEP-0107 User Mood XEP-0107.py, rc2 ✓✓✓ 0107-1 T×S+own rerun none
XEP-0108 User Activity XEP-0108.py, rc2 ✓✓✓ 0108-1 T×S+own rerun none
XEP-0118 User Tune XEP-0118.py, rc2 ✓✓✓ 0118-1 T×S+own rerun none
XEP-0158 CAPTCHA Forms XEP-0158.py, develop ✓✓✓ (S5 fixed) 0158-1 T×S rerun a proof of work from a room T is joining whose from field names another room (not solved)
XEP-0201 Message Threads XEP-0201.py, develop ✓✓✓ (M3 fixed) 0201-1 T×S rerun none
XEP-0232 Software Information WIP, XEP-0232.py, rc2 ✓✓✓ 0232-1 T×S rerun none
XEP-0277 Microblogging WIP, XEP-0277.py, develop ej ✓ ai ✓, pr rc2 ✓ 0277-1 T×S rerun (pr first) none
XEP-0298 Coin WIP, XEP-0298.py, na ×3 (no call headless) 0298-1 T(remote)×S now with a call: S as focus (<conference-info isfocus/> in its accept), full, partial and deleted documents merged by version, taken only from the call's peer, calls conference_info; a 200 KB document
XEP-0358 Jingle Publishing WIP, XEP-0358.py, develop ✓✓✓ (C1 fixed) 0358-1 T×S rerun S sends <start/>: record the refusal (row: not served)
XEP-0364 OTR XEP-0364.py, develop ✓✓✓ (S9 fixed) 0364-1 T×S rerun; 0364-2 T×T2 (new) T×T2: SMP with the right secret, a wrong one and an abort (S's engine has no SMP maths)
XEP-0394 Message Markup XEP-0394.py, rc2 ✓✓✓ 0394-1 T×S rerun none
XEP-0426 Character Counting XEP-0426.py, develop ✓✓✓ (C3 fixed) 0426-1 T×S rerun none
XEP-0439 Quick Responses XEP-0439.py, rc2 ✓✓✓ 0439-1 T×S rerun none
XEP-0453 DOAP no scenario 0453-1 offline, recorded on one server (see open point 4) python3 tools/doap.py --check on the tested commit; schema/tacky.doap parses as RDF; one SupportedXep per XEP row with its status and version
XEP-0463 MUC Affiliations Versioning WIP, XEP-0463.py, develop ej ✓ ai ✓, pr rc2 ✓ 0463-1 T×S rerun (pr first) a since T never reached forgets the list (the next join asks for all)
XEP-0470 Pubsub Attachments WIP, XEP-0470.py, develop ✓✓✓ (C10 fixed) 0470-1 T×S rerun none
XEP-0472 Pubsub Social Feed WIP, XEP-0472.py, develop ✓✓✓ 0472-1 T×S rerun none
XEP-0477 Pubsub Targeted Encryption XEP-0477.py, rc2 ✓✓✓ 0477-1 T×S+own rerun an <encrypted by/> naming another JID than the publisher is refused; the OMEMO 2 mechanism has no public method (not checked)
XEP-0483 HTTP Online Meetings WIP, XEP-0483.py, develop ej ✓ ai ✓, pr rc2 ✓ 0483-1 T×S rerun (pr first) none (#1 S1 regression stays in it)
XEP-0488 MUC Token Invite WIP, XEP-0488.py, rc2 ✓✓✓ (no server hands out tokens) 0488-1 T×S rerun optional on the fake (H5): a room that hands out tokens, <expired-token/>, a refusal written as a result with <error/>
XEP-0491 WebXDC WIP, XEP-0491.py, rc2 ✓✓✓ 0491-1 T×S rerun an update of 128 KiB + 1 is not filed
XEP-0501 Pubsub Stories WIP, XEP-0501.py, develop ej ✓ ai ✓, pr on fix-M7 ✓ 0501-1 T×S rerun (pr first) none
XEP-0503 Server-side Spaces WIP, XEP-0503.py, rc2 ej ✓ ai ✓, pr na 0503-1 T×S rerun none; pr keeps its na reason if the service still lacks the §4 features
XEP-0507 Jingle Content Category WIP, XEP-0507.py, one na run (pr) 0507-1 T(remote)×S now on all three: the scenario's step 5 (T accepts, set_screen_share → description-info slides, back → speaker); a description-info with anything beside categories → unsupported-info
XEP-0518 Payment Required WIP, XEP-0518.py, develop pr ✓ ej ✓, ai rc2 na 0518-1 T×S rerun (ai first) none

7d, GUI peers (anotherim-js in the VMs)

aim = anotherim-js; one scenario is one run per peer and server. Read vms/AGENTS.md first. Both VMs sit behind QEMU user NAT as 10.0.2.15, so a direct SOCKS5 candidate never connects; aim's XML log lags a click by up to 30 s.

Rows Scenario Peers Pass when Known before
XEP-0234, 0260 d-ft-1 aim sends a picture with "Send files directly" on, plain chat; then the peer sends one Dino, anotherim-web the offer names s5b (Dino) or the data channel (web), the proxy candidate is used and activated, the picture shows on both, hash verified, no upload pass on pr and ai with Dino; ej fails (P6: proxy65 at 127.0.0.2, then P5: Dino rejects the IBB replace)
XEP-0261 d-ft-2 the same on ejabberd, where SOCKS5 comes to nothing Dino, anotherim-web transport-replace to IBB accepted, bytes intact Dino: peer fail P5 expected, recorded as --cause peer; web is the cell that can pass
XEP-0343, 0391 d-ft-3 aim ↔ web in an OMEMO chat anotherim-web data channel, <security/> JET-OMEMO 2, sealed fingerprints, the picture decrypted on both plain data channel passes on all three
XEP-0293, 0294 d-call-1 audio then video call with vms/examples/netem.py adding 5 % loss in the video call Dino, anotherim-web both session descriptions carry rtcp-fb and rtp-hdrext and the answer is a subset of the offer; rtp.log shows RTP with the extension bit set and RTCP feedback packets (PT 205 or 206) during the loss calls pass with web; Dino P3 (no audio now and then) is a peer fail
XEP-0231, 0264 d-thumb-1 a picture each way, plain and OMEMO Dino, Gajim, Kaidan, anotherim-web the receiver shows the thumbnail before any download (screenshot before the click); aim's share has the cid and the data: thumbnail (1.2.5) Dino showed a download button for cid-only thumbnails before 1.2.5
XEP-0334, 0428 d-hint-1 typing, a reaction, a retraction, a reply, a file with a caption each way; then a second device of each account reads the archive Gajim, Dino no stored chat state; the reaction and the retraction are in the archive; no doubled quote on either side; a client without replies shows the fallback text –
urn:xmpp:mucsub:0 d-sub-1 on ejabberd and anotherim-server: aim follows a room without joining (first check that the app has a control for it; if not, na with that reason), the peer writes, reacts, corrects and retracts; aim answers; aim restarts Gajim, Dino aim shows the room and its unread count without being an occupant (the peer's occupant list lacks it), its message arrives, the subscription is there after the restart Prosody [na]
XEP-0215, 0176 d-nat-1 each VM in a network namespace with outbound UDP dropped except to the lab's TURN server; audio and video call; then UDP dropped altogether Dino, anotherim-web the <services/> answer has the TURN service with credentials, both sides offer relay candidates, the selected pair is relay–relay, tones both ways; without UDP the call uses TURN over TCP or fails with a reason, not a hang coturn on Prosody since #13

RFC rows: seeds and oracles for 7c

The RFC rows have runs; these are the cases the 7c proxy injects, and what counts as a wrong decision. Each finding becomes a repro-<id>.py.

Spec Injected by the proxy Wrong decision
RFC 6120 features without <starttls/>; a stream header with another from, no id, version 0.9; a bind result with another JID; <see-other-host/> to a foreign host; a stream error after every nth stanza; an unknown top-level element; an IQ result under a pending id with another from a plaintext login; a session under the wrong JID; a redirect followed without the certificate matching the account's domain; a stanza replayed in a loop (#1 C21); a spoofed IQ answer taken
RFC 6121 a roster push with a contact's from; presence floods; a subscription approved nobody asked for the push applied (row: only no from, our bare JID or our domain)
RFC 7590 a certificate for another domain; TLS 1.1 only; the connection cut inside the handshake a connection kept; a fallback to plaintext
RFC 7622 (WIP) JIDs with forbidden code points, a final dot, full-width forms, mixed case, in from and in roster items two chats for one address; a JID that never leaves the store (NIGHT 2026-11-05)
RFC 7677, XEP-0474, XEP-0440 a server-first whose nonce does not extend the client's; a wrong server signature; an iteration count of 1; -PLUS mechanisms removed from the list conn <State> connected in any of these
RFC 7395 <open/> with another from; see-other-uri to ws:// the redirect followed
XEP-0198 <a h/> above what was sent; <resumed/> with another previd; <failed/> mid-resume stanzas lost or sent twice; a session bound to another id
XEP-0280, 0313, 0359 carbons and MAM results with a contact's from; a <stanza-id by='our JID'/> written by the peer; a <fin/> under another query id a row stored from a forged wrapper (#1 S12); a message filed under a forged id
XEP-0478 stanzas one byte over the advertised limit, both ways a crash, or an answer dropped without policy-violation (NIGHT 2026-11-05)

0. Before start

  • Check upstream: libtacky develop, anotherim-js develop, and new comments on #1, #9, #24 and #25.
  • Build the newest tackyd-json and its HANDBOOK, plus the anotherim-js AppImage and web build if develop has moved.
  • Branches: ai/batch-7 (headless: 7a, 7b, 7c) and vm/batch-7 (7d). One PR each into main; results go here.

7a. Headless scenarios for the untested rows

tackyd-json against slixmpp, on Prosody, ejabberd and anotherim-server, one agent per server.

  • P0:
    • XEP-0166, 0176 and 0320 (Jingle, ICE, DTLS fingerprint), run tackyd against tackyd because slixmpp has no Jingle.
    • XEP-0420 SCE, checked inside the OMEMO 2 runs.
  • P1, protocol: 0048, 0050, 0060, 0066, 0068, 0082, 0203, 0221, 0231, 0260, 0261, 0264, 0292, 0293, 0294, 0300, 0317, 0334, 0338, 0339, 0343, 0367, 0372, 0380, 0385, 0391, 0398, 0428, 0434, 0446, 0450, 0452, 0469, urn:xmpp:mucsub:0, urn:fluux:poll:0.
  • P1, tacky-only: urn:tacky:sealed:1, omemo-link:0, storage:0, service:0, spans:0, live-location:0 and push-hint:0. These run libtacky against libtacky, so they catch crashes and regressions, not misread specs.
  • P2, if time allows: 0047, 0055, 0065, 0080, 0107, 0108, 0118, 0158, 0201, 0232, 0277, 0298, 0358, 0364, 0394, 0426, 0439, 0453, 0463, 0470, 0472, 0477, 0483, 0488, 0491, 0501, 0503, 0507, 0518.
  • Per row: the main path, the spec's MUST edge cases, and one malformed or unauthorised case, unless the feature list above says otherwise.
  • Harness change: run.toml can name extra specs (also = [...]), so the existing call and OMEMO runs also count for the Jingle and SCE rows they exercise.
  • Size: about 200–250 runs.

7b. libtacky code coverage

  • Build tackyd-json instrumented as in libtacky's coverage.mk: Rust -C instrument-coverage, C --coverage, and Tcl's procedure record.
  • Rerun all headless runs once per server with that build, and merge the results with libtacky's tools/cov-union.py and stats.py.
  • Output:
    • COVERAGE.md: per module, the lines interop reached next to the lines libtacky's own tests reached.
    • A list of the large regions nothing reached, mapped to features. These become new scenarios.

7c. Fuzzing as a hostile server

  • A proxy between tackyd-json and each server mutates real stanzas, seeded from the wire logs in runs/:
    • stanzas dropped, duplicated or reordered
    • bad attributes, oversized or deeply nested XML, wrong namespaces
    • forged from/to
    • stream errors mid-flight, slow reads
  • What it detects:
    • crashes, and hangs (a watchdog on the tacky calls)
    • memory growth
    • store corruption (the store is read back afterwards)
    • wrong security decisions (forged carbons, MAM results or stanza-ids accepted)
  • Seeds are fixed. Every finding gets a minimal repro scenario. The time box is 6 h per server, also run once under the 7b build.

7d. VM part (GUI peers, anotherim-js)

  • Rows that need real clients:
    • Jingle file transfer (0234, 0260, 0261) with Dino and Gajim
    • call features (0293, 0294)
    • thumbnails (0231, 0264)
    • message hints (0334, 0428)
    • MUC/SUB against ejabberd
  • Calls behind NAT: VM network namespaces with a TURN server in the lab, covering the XEP-0215 path.
  • One agent per server, as in batch 6.

7e. Hand test (after the runs)

Once the runs are finished, a checklist with step-by-step instructions goes here, for a person to run in the VMs. It covers:

  • the P0 cells on each server: login, roster, 1:1, MUC, OMEMO 2, MAM, file transfer and calls
  • anything batch 7 found that an agent couldn't judge

Each step gives the VM command, the accounts, what to click, what to expect and where to look (screenshot, log line). Those cells are what let FEATURES.md show VERIFIED.

Future (not in batch 7)

  • A fourth server (Openfire or Tigase): about 35% more runs. Planned for a later batch.
  • Push through FCM/APNs and the mobile clients: they don't run in the NixOS VMs; only a mock push app server would be possible.
  • Gateways and bridges (XEP-0100, Matrix, Telegram, Signal) and MatrixRTC: need external networks or self-hosted bridges.
  • Real Tor/I2P (RFC 1928): the SOCKS layer is already covered; real onion routing can't be tested offline.
  • MIX (0369, 0403–0408): no server or peer support worth testing yet.

Results

Filled in per part when its PR is merged.

🤖 Generated with Claude Code

Batch 7: cover what the harness has never run, measure how much of libtacky it reaches, and attack it with hostile server input. Starts after batch 6 (#25) is merged. At the start, 98 of the 208 rows of libtacky's [docs/features.md](https://dev.another.im/another-im/libtacky/src/branch/develop/docs/features.md) show no run in [FEATURES.md](FEATURES.md), most because they were `ai_ok = false` (agents' runs of those were hidden); 11 have no scenario at all. Only 3 cells were run by a person. **Every finding of batch 7, fuzzer included, is recorded in this issue**, not in #1 or #9. ## Feature list per spec (scenarios) Written 2026-11-11 from libtacky develop `65a31465` (`docs/features.md`, `HANDBOOK.md`, `docs/protocol/`), interop `main` (`features/*.toml`, `tools/ai/scenarios/`, `runs/`, `AI-RESULTS.md`, `tools/ai/NIGHT-2026-11-05.md`, `vms/AGENTS.md`) and #1, #9, #24, #25. One bullet with an id (`0166-2`) is one scenario, which is one run per server. Each scenario holds the main path, the MUST edge cases and at least one malformed or unauthorised case, as 7a asks. **Names used below:** T = tackyd-json under test (`ai{N}a`); S = slixmpp (`ai{N}b`); T2 = a second tackyd as the peer (`ai{N}b`, a store of its own); T' = a second tackyd device of T's own account; own = a slixmpp session of T's account; D1, D2 = slixmpp-omemo devices made for the run; fake = a local server on 127.0.0.1 as in `XEP-0215.py`; pr / ej / ai = Prosody, ejabberd, anotherim-server. Section numbers are quoted from the existing `RULES` lists or from libtacky's source. Where a line gives none, the author takes it from the spec (step 0 of `tools/ai/PROMPT.md`), never from memory. ### What the earlier batches already did (read before writing anything) - **"No run" in FEATURES.md does not mean "never run".** FEATURES.md leaves out low-confidence runs (🔸, `ai_ok = false`). Of the rows 7a lists, only 11 have no scenario and no run: XEP-0068, 0176, 0260, 0261, 0293, 0294, 0320, 0338, 0339, 0343, 0453. Every other row has a scenario in `tools/ai/scenarios/`, a `RULES` list, filled steps in `features/<slug>.toml` and runs on all three servers. Do not write those again: rerun them on the batch's build and add what is listed here. - **Many of those runs are on v1.2.0-rc2**, not on develop. "rc2" below marks them; they are rerun unchanged first, so a change since rc2 shows as a difference. - **Headless tackyd-json has no native media backend.** `media status` of the interop build answers `backend ""` ("neither webrtc nor datachannel is in this build"), so `calls start` never reaches a session-initiate; that is why XEP-0166 part C, XEP-0298 and XEP-0507 ended as INFO or `na`. The `remote` backend opens in the same binary (`--media-backend remote`: audioCalls, videoCalls, dataChannel, renegotiate, trickleIce; checked on `tackyd-json-develop-65a31465`). With it the scenario plays the page: it answers `calls <MediaOpen>` with a canned SDP (`calls local_description`, `local_candidate`, `media_state`) and reads `calls <MediaRemote>` / `<MediaCandidate>`. All Jingle call scenarios below use it. - **spans:0, live-location:0 and service:0 are read-only in libtacky** (rows: "Not sent", "read"). Run libtacky against libtacky and nothing crosses. For these three S stays the sender; the libtacky-against-libtacky part is the second device and the archive. - **Gajim 2.6 has no Jingle file transfer** (`clients.toml` `lacks`: XEP-0234, 0260, 0261). 7d names Dino and Gajim for it; only Dino and anotherim-web can run those cells. ### Rules for every scenario (mistakes made before) - **Server behaviour is INFO, never libtacky's fail.** Known differences are in `tools/ai/differences.toml`: ejabberd's mod_caps and Prosody's mod_pep send their own disco queries (count only the client's full JID); Prosody delivers each PEP notification twice; anotherim-server archives a message to a blocked JID before bouncing it. A fail that is the server's gets `--cause server` (XEP-0065 on Prosody was relabelled for this). - **A behaviour libtacky's row documents is not a bug** (C12, XEP-0153 photo hash before the vCard). Quote the row in the notes instead. - **The store outlives the run.** Make every body, node, room, feature name and device unique to the run; read and restore every setting touched (OMEMO switch and mode, `file_transfer_p2p`, `attachment_autofetch`, transports, `auto_trust`, blind trust). Take run-made OMEMO devices off the device lists and delete their bundles. RFC 7622's junk chats and XEP-0478's extra device stayed in the stores for days. - **iq ids restart with each tackyd session.** Scope every lookup with `since=mark()`. - **`Tacky.send_raw` goes through the debug tap and is counted by stream management** (XEP-0198 on ejabberd resent it). Do not use it around a resume or a stream error. - **Discover, do not hard-code:** MUC, pubsub, upload and proxy services by disco. Print what depends only on the server as INFO. - **Wait long enough for big stanzas:** ejabberd delivered an 88 KB stanza after about 15 s. - **Shell and disk:** `tools/ai/aish`, never `nix develop .#ai` (it filled the disk twice). Scratch in `mktemp -d`; delete export folders that hold history. - **Secrets:** never print or store passwords, TURN credentials, FAST tokens, invite tokens, registration passwords or an `account link_device` URI (it holds `osec`). Check the base64 forms too. Room passwords from config forms appeared in clear in `peer-slixmpp.log` once. - **slixmpp 1.17:** no Jingle (raw IQs), `xep_0202` cannot answer time queries, plugins through `p.xmpp.plugin[...]`. OMEMO 2 is python-omemo (slixmpp-omemo). - **A run the server restart interrupted** is finished `na` and redone. A run a revised scenario replaced is finished `na` with "superseded by <run>". - **Low confidence wording:** notes begin `low confidence: checked …; not checked: …`. ### Harness changes these scenarios need (before 7a starts) | # | Change | Needed by | |---|---|---| | H1 | `Tacky(..., media_backend="remote")` (today `--media-backend auto` is fixed in `tacky.py`) and a `Page` helper in `common.py`: canned SDP offers and answers (audio, audio+video, data channel), candidates, `media_state`, a byte pipe for `<MediaData>` / `data_message`. Fixtures: libtacky's `tests/client/test_calls_remote.tcl` (`CHROME_OFFER`, `CHROME_CANDIDATE`). | every call and data-channel scenario | | H2 | A second tackyd in one run: its own `log_name` (`peer-tackyd.log`), events log (`client-events.log` is fixed today, so two instances would share it) and debug log; `interop start --under-test tackyd --client tackyd` for a `kind = "peer"` feature. | every T×T2 scenario | | H3 | A second store for the same account (`Tacky(state_dir=…)`), for T'. | sealed:1, omemo-link:0, XEP-0450 | | H4 | `also = [...]` in run.toml (already in 7a). A run that fails one `also` spec's rules is rerun split, so one spec's bug does not fail the others' rows. | call and OMEMO runs | | H5 | A shared fake server (`tools/ai/fake.py`: direct TLS on 127.0.0.1, pinned certificate, PLAIN, programmable disco and answers), from `XEP-0215.py` and `RFC-5929.py`, which each carry a copy. | XEP-0048, 0068, 0452, 0488, storage:0 | ### Open points for the maintainer Decided 2026-11-11: 1. **Every row may count:** `ai_ok = true` in every `features/*.toml`. VERIFIED stays a person's pass; rows with a display or media part are still run `confidence = "low"`. 2. **XEP-0452 on the real prosody.asxp.io**, no fake server: the maintainer enables `mod_muc_mention_notifications` there (on the MUC component: `modules_enabled = { "muc_mention_notifications" }` and `muc_mmn_notify_unaffiliated_users = true`). `0452-2`'s fake part is dropped; its hostile cases go into `0452-1` on Prosody. ejabberd and anotherim-server stay `[na]`. 3. **urn:tacky:storage:0:** the fake server (`tools/ai/fake.py`) is its only run. 4. **XEP-0453:** the one offline check `0453-1`, recorded on Prosody; ejabberd and anotherim-server have an `[na]` reason. 5. **Run count:** 90 scenario ids (P2 has 31 with `0364-2`), about 260 runs, about 170 without P2. Harness H1–H5 and these decisions are in the batch 7 harness PR (branch `ai/batch-7-harness`). ### P0 **XEP-0166 Jingle** · has `XEP-0166.py` (R1–R17, file session over IBB), develop: pr ✓ ej ✓ ai ✓ (C31 fixed) - `0166-1` T×S, rerun, then add to the same file session: content-modify (answered, senders unchanged), content-remove (ends the session), description-info and security-info (unsupported-info), a responder's transport-replace crossing T's own (`<conflict/>` + `<tie-break/>`). R16 (a refused sid sent again) becomes a check, not INFO. Hostile: `initiator` naming a third JID; a session-initiate from a bare JID; a sid of 4 KB with markup characters; a 9th open incoming session (XEP-0234 row: `resource-constraint`). - `0166-2` T(remote)×S, raw Jingle, call session (new). T `calls start`, S answers the XEP-0353 proceed, the page gives the offer: session-initiate has `initiator`, every `<content/>` one description and one transport. S session-accept → `<MediaRemote type answer>`, `media_state connected` → `<Active>`. Then: S's transport-replace (refused; record the condition), S's content-add video → `<MediaRemote offer>`, the page's answer → content-accept, `<Renegotiated>`; S's content-remove; T's own content-add (a mid-call `local_description` offer), S content-reject → `<Renegotiated rejected>`; crossing session-initiates (tie-break); session-terminate both ways, then unknown-session. Hostile: session-accept from S's other resource; a second session-accept; `media_failed` from the page (session-terminate with a reason, `<Failed>`); a propose while a call is up (`busy`, `<Ended>` missed). - `0166-3` T(remote)×T2(remote), both ways (new): audio call, video call, reject, hang up from each side, a call to an offline peer. Pass when every action is acked, both sides end in the same state, and the SDP that reaches the far page has every line class the near page sent (codecs, rtcp-fb, extmap, ssrc, group, fingerprint, candidates). `also = [XEP-0167, XEP-0353]`. **XEP-0176 Jingle ICE-UDP** · no scenario - `0176-1` T(remote)×S (new). T's initiate carries `ufrag` and `pwd` from the page's SDP; every trickled transport-info carries both (libtacky cites XEP-0176 §5.3) and a full candidate (component, foundation, generation, id, ip, port, priority, protocol, type; rel-addr and rel-port for srflx and relay). S's candidates sent before the page opened are buffered and replayed as `<MediaCandidate>` after `<MediaRemote>`. A candidate under the known ufrag without a pwd is applied. New ufrag and pwd together are an ICE restart: on `remote` (`iceRestart` false) record what T does as INFO. TCP candidates are not signalled: S's `protocol='tcp'` candidate and the page's tcp candidate cross neither way. Hostile: priority 2^32, port 70000, an ip that is none, a ufrag with spaces or 300 characters, a candidate for an unknown content, 500 candidates at once, a transport-info from another resource. Pass when each is refused or dropped, no `<MediaCandidate>` carries it, and the session lives. - `0176-2` T(remote)×T2(remote) (new): trickle both ways with scripted host, srflx and relay candidates; `<MediaOpen>`'s `ice_servers` equal `extdisco fetch` on that server (stun and turn on all three since #13). `also = [XEP-0215]`. **XEP-0320 DTLS-SRTP in Jingle** · no scenario - `0320-1` T(remote)×S (new). T's initiate has `<fingerprint xmlns='urn:xmpp:jingle:apps:dtls:0' hash setup/>` in every transport, equal to the page's `a=fingerprint` and `a=setup`; S's accept with `setup='active'` reaches the page so; as responder the page's answer role is in T's session-accept. `calls <Security>`: `dtls_local`, `dtls_remote`, 7 SAS indices, `encryption none`. Hostile: an accept with no fingerprint; a fingerprint that is not colon-separated hex; `setup='bogus'`; two contents with different fingerprints; a fingerprint changed by a later transport-info. Pass when the malformed ones never reach the page and the call ends with a reason, not `<Active>`. - `0320-2` T(remote)×T2(remote), OMEMO on, both devices trusted and advertising `urn:tacky:jingle:dtls-omemo:0` (new): the fingerprints cross sealed as `docs/protocol/jingle-dtls-omemo-0.md` has it, `<Encryption>` / `<Security>` say `omemo` with the device and its trust, both sides show the same SAS. Then T2's device undecided: the trust says so. Hostile (S in a third session): S replays T2's sealed element in a session of its own → `<Failed>`, session-terminate `security-error`. `also = [urn:tacky:jingle:dtls-omemo:0]`. **XEP-0420 Stanza Content Encryption** · has `XEP-0420.py` (R1–R19), develop: pr ✓ ej ✓ ai ✓ (S2 fixed) - `0420-1` T×S+D1, rerun, then add what 1.2.0–1.2.4 brought: a retraction in an OMEMO 2 chat is sealed (`<retract/>`, fallback body and `<fallback/>` inside, no body beside), a bare `<retract/>` beside the ciphertext, none with `retract_hint` 0; a `<time/>` more than 30 days before a stamp the server vouches for (D1 sends while T is offline, T catches up) is refused `<DecryptFailed>` `sce`; an OMEMO 2 room message without `<to/>`, or with another room's, is refused. R19 stays INFO for receipts, markers and reactions (no server advertises `urn:tacky:push:sealed-signals:0`, #9 S15). - `0420-2` T×T2, OMEMO 2 both (new): body, reply, correction, retraction, reaction, file share, in a 1:1 chat and in a members-only non-anonymous room. Pass when no `<DecryptFailed>` appears and R6–R8 hold on every stanza of both logs. A helper `check_sce(stanza)` in `common.py` runs the same rules in every OMEMO 2 run (XEP-0384, 0434, 0450, 0454, jet-omemo:2), which then carry `also = [XEP-0420]`. ### P1, protocol **XEP-0048 Bookmarks** · has `XEP-0048.py` (R1–R10), rc2: ✓✓✓, R9–R10 never ran (all three servers have PEP bookmarks) - `0048-1` T×S+own, rerun. - `0048-2` T×fake without PEP bookmarks (new, H5): R9, R10: private XML fetched on connect, the whole `<storage/>` written back with other conferences, `<url/>` and foreign children unchanged. Hostile: 5000 conferences, a malformed jid, `autojoin='maybe'`, the same jid twice. **XEP-0050 Ad-Hoc Commands** · has `XEP-0050.py` (R1–R15), develop: ✓✓✓ (M2 fixed) - `0050-1` T×S, rerun, add: a required field left empty is refused before any IQ (row: validated before submit); the default action when `<actions/>` names no `execute`; the responder's `bad-action`, `bad-sessionid` and `session-expired` reach the caller. Hostile: the sessionid changed mid-session; a result from another JID under the same iq id; a 500-field form; an unknown note type. **XEP-0060 Publish-Subscribe** · has `XEP-0060.py` (R1–R16), `repro-C11.py`, develop: ✓✓✓ - `0060-1` T×S, rerun the full scenario on all three (ej and pr last ran only the C11 repro), add: subscriptions survive a restart of T; a `pending` or `unconfigured` subscribe answer is reported; a notification from the right service with a node T never subscribed to, and from another service with the right node, is no `pubsub <Notification>`. Hostile: an event with 1000 items; a 200 KB payload; `<delete><redirect/></delete>`. Not checked (row): §6.3, §8.7, §8.8, collections. **XEP-0066 Out of Band Data** · has `XEP-0066.py` (R1–R8), rc2: ✓✓✓ - `0066-1` T×S, rerun, add the 1.2.2 and 1.2.5 rules: a room occupant's picture is not fetched by itself; `contacts` means subscription both (a from-only contact's file waits); a stored `everyone` reads back as `contacts`. Hostile: a url with userinfo, of 4 KB, `aesgcm://` with a broken fragment, https to a local listener (count connections). **XEP-0068 Field Standardization** · no scenario - `0068-1` T×S+fake (new). Out: every `x:data` submit T sends in the run carries the right `FORM_TYPE`: MAM query, push enable, pubsub node config and publish-options, MUC config and `muc#register`, a CAPTCHA answer, an ad-hoc stage (the responder's own). In: on the fake, an upload form whose `FORM_TYPE` is missing, not hidden, or another value gives no `max-file-size` (storage-0 §5); a CAPTCHA with a wrong `FORM_TYPE` is not told (XEP-0158 R2); a caps answer with two forms of one `FORM_TYPE` is not cached (XEP-0232 R7). **XEP-0082 Date and Time Profiles** · has `XEP-0082.py` (R1–R8), rc2: ✓✓✓ - `0082-1` T×S+own, rerun, add malformed forms: `23:59:60`, year 0000 and 10000, lower-case `t` and `z`. **XEP-0203 Delayed Delivery** · WIP · has `XEP-0203.py` (R1–R10), rc2: ✓✓✓ - `0203-1` T×S+own, rerun, add: after a XEP-0198 resume a resent stanza's server stamp is taken, never earlier than that session's start; a forged `from=DOMAIN` stamp 16 s after login with no inbound traffic gets the arrival time (the window closed). o3 stays INFO (the row's open issue). **XEP-0221 Data Forms Media Element** · has `XEP-0221.py` (R1–R11), develop: ai ✓; pr ✗ and ej ✗ are rc2 runs (S5, fixed in 1.2.1) - `0221-1` T×S, rerun on pr and ej first: their cells still show the rc2 fail. Add: a `sha256+` cid; 100 `<uri/>`s in one `<media/>`; width 65536. **XEP-0231 Bits of Binary** · has `XEP-0231.py` (R1–R12), rc2: ✓✓✓ - `0231-1` T×S, rerun, add: a cid with max-age 2 is fetched again after 3 s; a result for T's BoB get from another JID is dropped; a fetched blob over the 64 KiB cap is refused; no fetch from a non-contact. **XEP-0260 Jingle SOCKS5 Bytestreams Transport** · no scenario (parts ran inside `XEP-0065.py` and `XEP-0234.py` part S) - `0260-1` T×S, S with the SOCKS5 client and server of `XEP-0065.py` (new). Candidates carry cid, host, jid, port, priority, type; `dstaddr` = SHA1(sid + initiator + responder); candidate-used and candidate-error both ways; both sides used: the higher priority wins, a tie goes to the initiator's; a proxy is activated by its offerer (`<activated/>`), `<proxy-error/>` moves to IBB by transport-replace; `mode='udp'` → `unsupported-transports`; `file <Update>` says `transport socks5`. Hostile: 12 candidates (trap listeners count: 8 dialled at most); candidates on 0.0.0.0, 224.0.0.1, 255.255.255.255, 127.0.0.1 (none dialled); candidate-used naming an unknown cid; `<activated/>` for a cid nobody used. - `0260-2` T×T2 (new): a 1 MiB file each way with transports `[socks5]`, direct, then through the server's proxy where it is reachable (ejabberd's proxy65 announces 127.0.0.2, #9 P6: INFO there). Pass when the bytes are intact and verified. **XEP-0261 Jingle IBB Transport** · no scenario (parts ran inside `XEP-0047.py`) - `0261-1` T×S (new). The responder's smaller `block-size` is honoured; the transport's `sid` is the `<open/>`'s; after both SOCKS5 sides report candidate-error T sends transport-replace to IBB and continues on S's transport-accept; as responder T accepts S's transport-replace to IBB. Hostile: transport-replace to an unknown namespace (transport-reject); block-size 0, 70000 and `abc` in the accept; a transport-accept nobody asked for (out-of-order). Not checked: the responder's replace when the account's proxy takes the data channel away. - `0261-2` T×T2 (new): transports `[ibb]`, 1 MiB each way, intact, `transport ibb`. **XEP-0264 Thumbnails** · has `XEP-0264.py` (R1–R10), `repro-C15.py`, develop: ✓✓✓ (ej and pr by the repro only) - `0264-1` T×S, rerun the full scenario on all three, add 1.2.5: an upload's SFS share names the thumbnail twice, by cid and as a `data:` URI when it is at most 16 KiB; between 16 and 64 KiB by cid only. The same inside an OMEMO 2 envelope is checked in `0420-2`. **XEP-0292 vCard4 Over XMPP** · has `XEP-0292.py` (R1–R11), `repro-C16.py`, develop: ✓✓✓ (ej and pr by the repro only) - `0292-1` T×S, rerun the full scenario, add: org, title, role, impp, tz, pronouns and adr round trip; two `vcard set` back to back on Prosody leave the second on the node (the retried publish that overwrote a newer one, NIGHT 2026-11-05); no echo of the server's converted item. **Check for #9 C47:** after `vcard set`, S reads T's vcard-temp: R10 says FN, N and NICKNAME are mirrored, the VM runs saw Kaidan read an empty card on ej and ai. Report which holds. Hostile: a 200 KB item; an item id that is not `current`; two items; a forged event from a full JID. **XEP-0293 RTP Feedback, XEP-0294 RTP Header Extensions, XEP-0338 Jingle Grouping, XEP-0339 Source-Specific Media** · no scenario - `0293-1` T(remote)×S, one run with `also = [XEP-0294, XEP-0338, XEP-0339]` (new). One SDP fixture, both directions (the page's SDP → T's Jingle; S's Jingle → `<MediaRemote>` SDP), a call T starts and a call S starts: - 0293: `a=rtcp-fb:PT nack pli`, `ccm fir`, `transport-cc`, `a=rtcp-fb:* …` and `trr-int` ↔ `<rtcp-fb type subtype/>` and `<rtcp-fb-trr-int value/>` in `urn:xmpp:jingle:apps:rtp:rtcp-fb:0`, under the payload-type or, for `*`, the description. Hostile: an `<rtcp-fb/>` without `type`; a non-numeric trr-int; 300 of them. - 0294: `a=extmap:ID[/direction] URI [params]` ↔ `<rtp-hdrext id uri senders/>` in `urn:xmpp:jingle:apps:rtp:rtp-hdrext:0`, the direction as `senders` for an initiator and for a responder, parameters kept, `a=extmap-allow-mixed` ↔ `<extmap-allow-mixed/>`. Ids: 1, 14, 15, 255, 4096 accepted; 0, 256, 4352, -1 and `x` refused (`jinglesdp.tcl`). - 0338: every `a=group` (BUNDLE and LS) ↔ `<group semantics><content name/></group>` in `urn:xmpp:jingle:apps:grouping:0`. A group naming a content the session lacks loses that name; semantics that is no token is refused. - 0339: `a=ssrc:ID name:value`, a valueless parameter and `a=ssrc-group:FID a b` ↔ `<source ssrc><parameter/></source>` and `<ssrc-group semantics>` in `urn:xmpp:jingle:apps:rtp:ssma:0`. ssrc 4294967295 accepted; 4294967296, -1 and `abc` refused. - Each spec's disco feature is in T's disco#info (INFO where the spec asks for none). Pass when every mapping is exact both ways and each refused input ends in an IQ error or a session-terminate with a reason, never a crash or a half-applied description. **XEP-0300 Hash Functions** · WIP · has `XEP-0300.py` (R1–R15), rc2: ✓✓✓ - `0300-1` T×S, rerun, add: a sha-256 value of 31 bytes; `algo='SHA-256'`. Not checked (row): BLAKE2b. **XEP-0317 Hats** · has `XEP-0317.py` (R1–R10), rc2: ej ✓ ai ✓ pr `na` (Prosody 13 runs no hats commands) - `0317-1` T×S, rerun, add: `<hats/>` in a message is not read; a 10 KB title. Prosody: R1 and R10 only, verdict `na` as before. **XEP-0334 Message Processing Hints** · has `XEP-0334.py` (R1–R11), rc2: ✓✓✓ - `0334-1` T×S+D1, rerun, add: a sealed retraction (1.2.4) carries `<store/>`; hints on a carbon and on an archived copy never drop the row. **XEP-0343 WebRTC Data Channels in Jingle** · no scenario - `0343-1` T(remote)×S (new). Transports `[datachannel, ibb]`: T's offer has the `urn:xmpp:jingle:transports:webrtc-datachannel:1` transport around ice-udp with `sctp-port` and `max-message-size`; `<MediaOpen data=1 role offer>`; after `data_state open` the file leaves as `<MediaData>` chunks and `<MediaDataClose>`; the bytes equal the file. S offers: `file accept`, the page feeds `data_message` chunks, the file is kept verified. `media_state failed` → transport-replace to the next transport (XEP-0260, 0261 rows). Hostile: more bytes than `<size/>`; `data_message` before `open`; a wrong hash (not kept, `<media-error/>`); a non-numeric `sctp-port` (5000 assumed). - `0343-2` T(remote)×T2(remote), the scenario pipes the bytes between the pages (new): a file each way in a clear chat, then in an OMEMO 2 chat (JET-OMEMO 2, fingerprints sealed). `also = [XEP-0391, urn:tacky:jingle:jet-omemo:2, urn:tacky:jingle:dtls-omemo:0]`. **XEP-0367 Message Attaching** · has `XEP-0367.py` (R1–R15), develop: ✓✓✓ (C6 fixed) - `0367-1` T×S+own, rerun, add: an attach-to naming its own message; A → B → A; 100 captions for one file; in a room, the same nick under another occupant id is not folded. **XEP-0372 References** · has `XEP-0372.py` (R1–R10), rc2: ✓✓✓ - `0372-1` T×S, rerun, add: 1000 references in one message; a mention after XML character references in the body. **XEP-0380 Explicit Message Encryption** · has `XEP-0380.py` (R1–R13), `repro-C17.py`, develop: ✓✓✓ (ej and pr by the repro only) - `0380-1` T×S+D1, rerun the full scenario, add #9 M27: the stored record of an OMEMO 2 message T sent (`raw_xml`) names `urn:xmpp:omemo:2`. Hostile: 50 `<encryption/>` elements; a 5 KB namespace; a `name` with markup and bidi overrides. **XEP-0385 SIMS** · has `XEP-0385.py` (R1–R18), `repro-C13.py`, develop: ✓✓✓ (ej and pr by the repro only) - `0385-1` T×S, rerun the full scenario, add: 30 sources; a size of -1 and of 2^64; a name with a path, a bidi override and a NUL. `legacy_file_metadata` 0 (#1 S3) stays in the XEP-0384 runs, which get `also = [XEP-0385]`. **XEP-0391 Jingle Encrypted Transports** · has `XEP-0391.py` (R1–R17), rc2: ✓✓✓ - `0391-1` T×S+D1, rerun, add 1.2.5 (#9 P4): Dino's cipher spelling is sent and both spellings are read; an offer of 256 MiB + 1 is refused before any byte. The libtacky-against-libtacky transfer is `0343-2`. **XEP-0398 Avatar to vCard Conversion** · has `XEP-0398.py` (R1–R6), rc2: ✓✓✓ - `0398-1` T×S, rerun, add: a vCard PHOTO of 5 MB, of type SVG, and an EXTVAL http url (nothing fetched); the conversion feature in a forged disco#info result is not believed. **XEP-0428 Fallback Indication** · has `XEP-0428.py` (R1–R10), develop: ✓✓✓ (C4 fixed) - `0428-1` T×S, rerun, add: 500 `<fallback/>` elements; start = end; overlapping and unsorted ranges (row: merged). **XEP-0434 Trust Messages** · WIP · has `XEP-0434.py` (R1–R13), rc2: ✓✓✓ - `0434-1` T×S+D1+D2, rerun, add: two `<trust-message/>` in one message (R1: exactly one); a usage other than ATM; a trust message in the clear; a contact's device naming a third JID as key owner (ignored). Not checked (row): OMEMO 0.3 keys, the archive. **XEP-0446 File Metadata** · has `XEP-0446.py` (R1–R12), rc2: ✓✓✓ - `0446-1` T×S, rerun, add: a `<size/>` that is not the bytes'; a negative `<length/>`; width 0 and 10^9; a 4 KB name; two `<file/>` elements. **XEP-0450 Automatic Trust Management** · WIP · has `XEP-0450.py` (R1–R13, R6 not run), rc2: ✓✓✓ - `0450-1` T×S+D1–D4, rerun, add R6 (a key not fetched yet) and the bounds: the 33rd trust message from a device not authenticated and the 257th unknown key are not kept. - `0450-2` T+T'×T2 (new, H3): T verifies T': T' gets every key T verified and refused; T's refusal of a T2 key goes to T' and not to T2; T2, verified, gets T's and T''s keys. Pass when the trust lists of T and T' are equal at the end. **XEP-0452 MUC Mention Notifications** · has `XEP-0452.py` (R1–R5), rc2: `na` ×3 (see open point 2) - `0452-1` T×S, rerun on Prosody only if the module is enabled; else `[na]` on all three. - `0452-2` T×fake (new, H5): R1, R3, R5 with the room's bare JID as the sender. Hostile: a forwarded message of another room; a non-groupchat one; a stamp in the future (never later than now); a 5000-character text (cut at 1024); the same mention twice (told once); a room T does not know. **XEP-0469 Bookmark Pinning** · has `XEP-0469.py` (R1–R7), rc2: ✓✓✓ - `0469-1` T×own, rerun, add: own changes the bookmark's name, then T pins: the pin's republish keeps the new name; `<pinned/>` twice; `<pinned/>` with children. **urn:xmpp:mucsub:0 ejabberd MUC/Sub** · has `urn-xmpp-mucsub-0.py` (R1–R13), develop: ej ✓ ai ✓, pr `[na]` - `mucsub-1` T×S, rerun, add: a kick, a ban and the room's destruction each end the stored subscription; S unsubscribes T while T is offline, and T's list is reconciled at connect; a reaction, a correction and a retraction through MUC/Sub land on the room chat's rows; the archive catch-up files the room's copy once. Hostile: a wrapper from the MUC service's domain; a wrapper from room A carrying a message `from='roomB/nick'`. **urn:fluux:poll:0 ProcessOne polls** · has `urn-fluux-poll-0.py` (R1–R27), develop: pr ✓ ej ✓, ai rc2 ✓ - `poll-1` T×S, rerun (ai first), add the room part: the tally by occupant id, a late joiner reading poll, votes and close from the archive (`{urn:xmpp:mam:2}ids`), `<poll-results/>` asked of the creator. Hostile: a results IQ from a non-occupant; 21 options; the same emoji twice; a deadline in the past; a close from the creator's nick under another occupant id. - `poll-2` T×T2 in a room (new): poll, votes, close and results between two libtackys. Pass when both show the same tally. ### P1, tacky-only These catch crashes and regressions, not misread specs: the spec is `docs/protocol/<name>.md` of the tested commit. **urn:tacky:sealed:1** · has `urn-tacky-sealed-1.py` (R1–R23), develop: pr ✗ ej ✗ (#1 M18 left: a replayed draft is dropped without a debug line), ai rc2 ✓ - `sealed-1` T×own, rerun on all three; say for M18 whether the item is refused (the security rule) apart from whether the line is logged. - `sealed-2` T+T' (new, H3): T' publishes its seal key and T trusts it: T seals the drafts, pins, tags and mute list again to T' under their own stamps, and T' reads them; a draft T' writes reaches T; T' set untrusted gets no key in the next item. Hostile (own): a seal key item signed by another identity; a key under a device id not on the OMEMO list. **urn:tacky:omemo-link:0** · has `urn-tacky-omemo-link-0.py` (R1–R15, the showing side only), rc2: ✓✓✓ - `link-1` T×own, rerun. - `link-2` T+T' (new, H3): T `account link_device`, T' (an empty store) `account add_from_uri`: `omemo <LinkVerified>` on both, each trusts the other's device. A link whose `ofp` is changed → `<LinkVerifyFailed>` on T'. The same link used by a third store verifies nothing (spent). The URI is a secret: never in a log, a note or the scenario's output. **urn:tacky:storage:0** · has `urn-tacky-storage-0.py` (R1–R12), rc2: `na` ×3 (see open point 3) - `storage-1` T×own, rerun; `[na]` stays while no server offers the service. - `storage-2` T×fake with an upload and a storage item (new, H5): R2, R6–R9, R12: `-personal 1` asks the storage service; `file keep` runs the `keep` command, an `aesgcm://` URL goes without its fragment and comes back with it. Hostile: `keep` answers an http url or another host's; a stage that never completes; two storage services (the first is used). **urn:tacky:service:0** · read-only · has `urn-tacky-service-0.py` (R1–R15), develop: ✓✓✓ (C5 fixed) - `service-1` T×S+own, rerun, add: the same rows after an archive catch-up (T offline while S sends); `<service/>` twice in one message; a 300-character kind; 200 attributes. **urn:tacky:spans:0** · read-only · has `urn-tacky-spans-0.py` (R1–R17), `repro-S6.py`, develop: pr ✓ ej ✓, ai rc2 ✓ - `spans-1` T×S+D1, rerun the full scenario on all three, add: spans inside an OMEMO 2 envelope read the same; spans beside a ciphertext are ignored (XEP-0420 R16). **urn:tacky:live-location:0** · read-only · has `urn-tacky-live-location-0.py` (R1–R11), develop: ej ✓, pr and ai rc2 ✓ - `live-1` T×S+own, rerun, add: moves that arrive from the archive after T was offline leave one row at the last place with no revision; a live location in a room is moved only by the same occupant id; `until` already past at the start. **urn:tacky:push-hint:0** · has `urn-tacky-push-hint-0.py` (R1–R9, in the clear only), rc2: ✓✓✓ - `hint-1` T×S, rerun. - `hint-2` T×T2, OMEMO 2 (new): an album with a caption: `<no-push kind='sources'/>` and `kind='caption'` stand beside the ciphertext; a caption that mentions someone or answers a message carries none. Not checked: what a push service does with it (no app server in the lab). ### P2, if time allows One scenario each; "rerun" means the existing scenario unchanged on the batch's build, with the adds in the same run. | Spec | Has, last result | Scenario | Add | |---|---|---|---| | XEP-0047 IBB | `XEP-0047.py`, develop ✓✓✓ (C2 fixed) | `0047-1` T×S rerun | the seq wrap 65535 → 0 (block-size 16, about 1.05 MB; time box 10 min, else not checked); either side closing mid-stream | | XEP-0055 Jabber Search | `XEP-0055.py`, rc2 ✓✓✓ | `0055-1` T×S rerun | `muc search` (1.2.0) against a room S plays | | XEP-0065 SOCKS5 Bytestreams | `XEP-0065.py`, rc2 ✓✓✓ (pr relabelled: mod_proxy65) | `0065-1` T×S rerun | nothing new; `0260-1` covers the transport; ej's proxy announces 127.0.0.2 (#9 P6) | | XEP-0080 User Location | `XEP-0080.py`, `repro-S8.py`, develop ✓✓✓ | `0080-1` T×S rerun the full scenario | none | | XEP-0107 User Mood | `XEP-0107.py`, rc2 ✓✓✓ | `0107-1` T×S+own rerun | none | | XEP-0108 User Activity | `XEP-0108.py`, rc2 ✓✓✓ | `0108-1` T×S+own rerun | none | | XEP-0118 User Tune | `XEP-0118.py`, rc2 ✓✓✓ | `0118-1` T×S+own rerun | none | | XEP-0158 CAPTCHA Forms | `XEP-0158.py`, develop ✓✓✓ (S5 fixed) | `0158-1` T×S rerun | a proof of work from a room T is joining whose `from` field names another room (not solved) | | XEP-0201 Message Threads | `XEP-0201.py`, develop ✓✓✓ (M3 fixed) | `0201-1` T×S rerun | none | | XEP-0232 Software Information | WIP, `XEP-0232.py`, rc2 ✓✓✓ | `0232-1` T×S rerun | none | | XEP-0277 Microblogging | WIP, `XEP-0277.py`, develop ej ✓ ai ✓, pr rc2 ✓ | `0277-1` T×S rerun (pr first) | none | | XEP-0298 Coin | WIP, `XEP-0298.py`, `na` ×3 (no call headless) | `0298-1` T(remote)×S | now with a call: S as focus (`<conference-info isfocus/>` in its accept), full, partial and deleted documents merged by version, taken only from the call's peer, `calls conference_info`; a 200 KB document | | XEP-0358 Jingle Publishing | WIP, `XEP-0358.py`, develop ✓✓✓ (C1 fixed) | `0358-1` T×S rerun | S sends `<start/>`: record the refusal (row: not served) | | XEP-0364 OTR | `XEP-0364.py`, develop ✓✓✓ (S9 fixed) | `0364-1` T×S rerun; `0364-2` T×T2 (new) | T×T2: SMP with the right secret, a wrong one and an abort (S's engine has no SMP maths) | | XEP-0394 Message Markup | `XEP-0394.py`, rc2 ✓✓✓ | `0394-1` T×S rerun | none | | XEP-0426 Character Counting | `XEP-0426.py`, develop ✓✓✓ (C3 fixed) | `0426-1` T×S rerun | none | | XEP-0439 Quick Responses | `XEP-0439.py`, rc2 ✓✓✓ | `0439-1` T×S rerun | none | | XEP-0453 DOAP | no scenario | `0453-1` offline, recorded on one server (see open point 4) | `python3 tools/doap.py --check` on the tested commit; `schema/tacky.doap` parses as RDF; one SupportedXep per XEP row with its status and version | | XEP-0463 MUC Affiliations Versioning | WIP, `XEP-0463.py`, develop ej ✓ ai ✓, pr rc2 ✓ | `0463-1` T×S rerun (pr first) | a `since` T never reached forgets the list (the next join asks for all) | | XEP-0470 Pubsub Attachments | WIP, `XEP-0470.py`, develop ✓✓✓ (C10 fixed) | `0470-1` T×S rerun | none | | XEP-0472 Pubsub Social Feed | WIP, `XEP-0472.py`, develop ✓✓✓ | `0472-1` T×S rerun | none | | XEP-0477 Pubsub Targeted Encryption | `XEP-0477.py`, rc2 ✓✓✓ | `0477-1` T×S+own rerun | an `<encrypted by/>` naming another JID than the publisher is refused; the OMEMO 2 mechanism has no public method (not checked) | | XEP-0483 HTTP Online Meetings | WIP, `XEP-0483.py`, develop ej ✓ ai ✓, pr rc2 ✓ | `0483-1` T×S rerun (pr first) | none (#1 S1 regression stays in it) | | XEP-0488 MUC Token Invite | WIP, `XEP-0488.py`, rc2 ✓✓✓ (no server hands out tokens) | `0488-1` T×S rerun | optional on the fake (H5): a room that hands out tokens, `<expired-token/>`, a refusal written as a result with `<error/>` | | XEP-0491 WebXDC | WIP, `XEP-0491.py`, rc2 ✓✓✓ | `0491-1` T×S rerun | an update of 128 KiB + 1 is not filed | | XEP-0501 Pubsub Stories | WIP, `XEP-0501.py`, develop ej ✓ ai ✓, pr on fix-M7 ✓ | `0501-1` T×S rerun (pr first) | none | | XEP-0503 Server-side Spaces | WIP, `XEP-0503.py`, rc2 ej ✓ ai ✓, pr `na` | `0503-1` T×S rerun | none; pr keeps its `na` reason if the service still lacks the §4 features | | XEP-0507 Jingle Content Category | WIP, `XEP-0507.py`, one `na` run (pr) | `0507-1` T(remote)×S | now on all three: the scenario's step 5 (T accepts, `set_screen_share` → description-info `slides`, back → `speaker`); a description-info with anything beside categories → unsupported-info | | XEP-0518 Payment Required | WIP, `XEP-0518.py`, develop pr ✓ ej ✓, ai rc2 `na` | `0518-1` T×S rerun (ai first) | none | ### 7d, GUI peers (anotherim-js in the VMs) aim = anotherim-js; one scenario is one run per peer and server. Read `vms/AGENTS.md` first. Both VMs sit behind QEMU user NAT as 10.0.2.15, so a direct SOCKS5 candidate never connects; aim's XML log lags a click by up to 30 s. | Rows | Scenario | Peers | Pass when | Known before | |---|---|---|---|---| | XEP-0234, 0260 | `d-ft-1` aim sends a picture with "Send files directly" on, plain chat; then the peer sends one | Dino, anotherim-web | the offer names s5b (Dino) or the data channel (web), the proxy candidate is used and activated, the picture shows on both, hash verified, no upload | pass on pr and ai with Dino; ej fails (P6: proxy65 at 127.0.0.2, then P5: Dino rejects the IBB replace) | | XEP-0261 | `d-ft-2` the same on ejabberd, where SOCKS5 comes to nothing | Dino, anotherim-web | transport-replace to IBB accepted, bytes intact | Dino: peer fail P5 expected, recorded as `--cause peer`; web is the cell that can pass | | XEP-0343, 0391 | `d-ft-3` aim ↔ web in an OMEMO chat | anotherim-web | data channel, `<security/>` JET-OMEMO 2, sealed fingerprints, the picture decrypted on both | plain data channel passes on all three | | XEP-0293, 0294 | `d-call-1` audio then video call with `vms/examples/netem.py` adding 5 % loss in the video call | Dino, anotherim-web | both session descriptions carry `rtcp-fb` and `rtp-hdrext` and the answer is a subset of the offer; `rtp.log` shows RTP with the extension bit set and RTCP feedback packets (PT 205 or 206) during the loss | calls pass with web; Dino P3 (no audio now and then) is a peer fail | | XEP-0231, 0264 | `d-thumb-1` a picture each way, plain and OMEMO | Dino, Gajim, Kaidan, anotherim-web | the receiver shows the thumbnail before any download (screenshot before the click); aim's share has the cid and the `data:` thumbnail (1.2.5) | Dino showed a download button for cid-only thumbnails before 1.2.5 | | XEP-0334, 0428 | `d-hint-1` typing, a reaction, a retraction, a reply, a file with a caption each way; then a second device of each account reads the archive | Gajim, Dino | no stored chat state; the reaction and the retraction are in the archive; no doubled quote on either side; a client without replies shows the fallback text | – | | urn:xmpp:mucsub:0 | `d-sub-1` on ejabberd and anotherim-server: aim follows a room without joining (first check that the app has a control for it; if not, `na` with that reason), the peer writes, reacts, corrects and retracts; aim answers; aim restarts | Gajim, Dino | aim shows the room and its unread count without being an occupant (the peer's occupant list lacks it), its message arrives, the subscription is there after the restart | Prosody `[na]` | | XEP-0215, 0176 | `d-nat-1` each VM in a network namespace with outbound UDP dropped except to the lab's TURN server; audio and video call; then UDP dropped altogether | Dino, anotherim-web | the `<services/>` answer has the TURN service with credentials, both sides offer relay candidates, the selected pair is relay–relay, tones both ways; without UDP the call uses TURN over TCP or fails with a reason, not a hang | coturn on Prosody since #13 | ### RFC rows: seeds and oracles for 7c The RFC rows have runs; these are the cases the 7c proxy injects, and what counts as a wrong decision. Each finding becomes a `repro-<id>.py`. | Spec | Injected by the proxy | Wrong decision | |---|---|---| | RFC 6120 | features without `<starttls/>`; a stream header with another `from`, no id, version 0.9; a bind result with another JID; `<see-other-host/>` to a foreign host; a stream error after every nth stanza; an unknown top-level element; an IQ result under a pending id with another `from` | a plaintext login; a session under the wrong JID; a redirect followed without the certificate matching the account's domain; a stanza replayed in a loop (#1 C21); a spoofed IQ answer taken | | RFC 6121 | a roster push with a contact's `from`; presence floods; a subscription approved nobody asked for | the push applied (row: only no `from`, our bare JID or our domain) | | RFC 7590 | a certificate for another domain; TLS 1.1 only; the connection cut inside the handshake | a connection kept; a fallback to plaintext | | RFC 7622 (WIP) | JIDs with forbidden code points, a final dot, full-width forms, mixed case, in `from` and in roster items | two chats for one address; a JID that never leaves the store (NIGHT 2026-11-05) | | RFC 7677, XEP-0474, XEP-0440 | a server-first whose nonce does not extend the client's; a wrong server signature; an iteration count of 1; -PLUS mechanisms removed from the list | `conn <State> connected` in any of these | | RFC 7395 | `<open/>` with another `from`; `see-other-uri` to `ws://` | the redirect followed | | XEP-0198 | `<a h/>` above what was sent; `<resumed/>` with another previd; `<failed/>` mid-resume | stanzas lost or sent twice; a session bound to another id | | XEP-0280, 0313, 0359 | carbons and MAM results with a contact's `from`; a `<stanza-id by='our JID'/>` written by the peer; a `<fin/>` under another query id | a row stored from a forged wrapper (#1 S12); a message filed under a forged id | | XEP-0478 | stanzas one byte over the advertised limit, both ways | a crash, or an answer dropped without `policy-violation` (NIGHT 2026-11-05) | ## 0. Before start - Check upstream: libtacky develop, anotherim-js develop, and new comments on #1, #9, #24 and #25. - Build the newest tackyd-json and its HANDBOOK, plus the anotherim-js AppImage and web build if develop has moved. - Branches: `ai/batch-7` (headless: 7a, 7b, 7c) and `vm/batch-7` (7d). One PR each into main; results go here. ## 7a. Headless scenarios for the untested rows tackyd-json against slixmpp, on Prosody, ejabberd and anotherim-server, one agent per server. - **P0:** - XEP-0166, 0176 and 0320 (Jingle, ICE, DTLS fingerprint), run tackyd against tackyd because slixmpp has no Jingle. - XEP-0420 SCE, checked inside the OMEMO 2 runs. - **P1, protocol:** 0048, 0050, 0060, 0066, 0068, 0082, 0203, 0221, 0231, 0260, 0261, 0264, 0292, 0293, 0294, 0300, 0317, 0334, 0338, 0339, 0343, 0367, 0372, 0380, 0385, 0391, 0398, 0428, 0434, 0446, 0450, 0452, 0469, urn:xmpp:mucsub:0, urn:fluux:poll:0. - **P1, tacky-only:** urn:tacky:sealed:1, omemo-link:0, storage:0, service:0, spans:0, live-location:0 and push-hint:0. These run libtacky against libtacky, so they catch crashes and regressions, not misread specs. - **P2, if time allows:** 0047, 0055, 0065, 0080, 0107, 0108, 0118, 0158, 0201, 0232, 0277, 0298, 0358, 0364, 0394, 0426, 0439, 0453, 0463, 0470, 0472, 0477, 0483, 0488, 0491, 0501, 0503, 0507, 0518. - **Per row:** the main path, the spec's MUST edge cases, and one malformed or unauthorised case, unless the feature list above says otherwise. - **Harness change:** run.toml can name extra specs (`also = [...]`), so the existing call and OMEMO runs also count for the Jingle and SCE rows they exercise. - **Size:** about 200–250 runs. ## 7b. libtacky code coverage - Build tackyd-json instrumented as in libtacky's `coverage.mk`: Rust `-C instrument-coverage`, C `--coverage`, and Tcl's procedure record. - Rerun all headless runs once per server with that build, and merge the results with libtacky's `tools/cov-union.py` and `stats.py`. - Output: - `COVERAGE.md`: per module, the lines interop reached next to the lines libtacky's own tests reached. - A list of the large regions nothing reached, mapped to features. These become new scenarios. ## 7c. Fuzzing as a hostile server - A proxy between tackyd-json and each server mutates real stanzas, seeded from the wire logs in runs/: - stanzas dropped, duplicated or reordered - bad attributes, oversized or deeply nested XML, wrong namespaces - forged from/to - stream errors mid-flight, slow reads - What it detects: - crashes, and hangs (a watchdog on the `tacky` calls) - memory growth - store corruption (the store is read back afterwards) - wrong security decisions (forged carbons, MAM results or stanza-ids accepted) - Seeds are fixed. Every finding gets a minimal repro scenario. The time box is 6 h per server, also run once under the 7b build. ## 7d. VM part (GUI peers, anotherim-js) - **Rows that need real clients:** - Jingle file transfer (0234, 0260, 0261) with Dino and Gajim - call features (0293, 0294) - thumbnails (0231, 0264) - message hints (0334, 0428) - MUC/SUB against ejabberd - **Calls behind NAT:** VM network namespaces with a TURN server in the lab, covering the XEP-0215 path. - One agent per server, as in batch 6. ## 7e. Hand test (after the runs) Once the runs are finished, a checklist with step-by-step instructions goes here, for a person to run in the VMs. It covers: - the P0 cells on each server: login, roster, 1:1, MUC, OMEMO 2, MAM, file transfer and calls - anything batch 7 found that an agent couldn't judge Each step gives the VM command, the accounts, what to click, what to expect and where to look (screenshot, log line). Those cells are what let FEATURES.md show VERIFIED. ## Future (not in batch 7) - **A fourth server (Openfire or Tigase):** about 35% more runs. Planned for a later batch. - **Push through FCM/APNs and the mobile clients:** they don't run in the NixOS VMs; only a mock push app server would be possible. - **Gateways and bridges (XEP-0100, Matrix, Telegram, Signal) and MatrixRTC:** need external networks or self-hosted bridges. - **Real Tor/I2P (RFC 1928):** the SOCKS layer is already covered; real onion routing can't be tested offline. - **MIX (0369, 0403–0408):** no server or peer support worth testing yet. ## Results _Filled in per part when its PR is merged._ 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
vm/batch-6
ai/batch-7-harness
vm/batch-5
report-rework
vm/batch-4
vm/batch-3
vm/batch-2
vm/batch-1
vm/batch-0
vms/run-provenance
vms/h4-h5-composer-profile
vms/h1-h6-devices-encryption
vms/h2-h3-calls-rooms
report/matrix-per-server
vms/consolidate
infra/prosody-turn
vms/plan
vms/runs-2
vms/harness-1-5
feat/client-vms
ci/xmllog-upstream
harness-cleanup
infra/second-runner
ai-runs-2026-11-07
ci/fast-check
ai-runs-2026-11-05
anotherim-v0.8.0-xmllog
anotherim-v0.8.0-rc10-xmllog
anotherim-v0.8.0-rc9-xmllog
anotherim-v0.8.0-rc7-xmllog
anotherim-v0.8.0-rc4-xmllog
anotherim-v0.8.0-rc3-xmllog
anotherim-v0.8.0-rc2-xmllog
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
another-im/interop#29
No description provided.