vms: multi-VM harness and 89 GUI-client runs on three servers #10

Merged
asxpi merged 105 commits from vms/harness-1-5 into main 2026-11-08 19:54:27 +00:00
Member

The VM harness grown for real interop runs, and the 89 runs it recorded on
2026-11-08 (anotherim-js 0.8.0-rc9 / libtacky v1.2.3 against Gajim, Dino
and Kaidan on prosody, ejabberd and anotherim-server; 77 pass, 12 fail).
Bugs found: #9.

Harness (vms/):

  • several VMs per driver run (--peer, nodes machine/peer/peer2,
    boot_all), disks per client and server so servers run in parallel
  • fake_av/hears (PipeWire null sink, sine on a fake mic, v4l2loopback
    camera, Goertzel), net_down/netem/freeze (+ sync_clock),
    expect(text=, log=), wait_notify (dunst + dbus-monitor)
  • chat helpers per client (open_chat, send, wait_message) and
    vm.py --record writing run dirs; examples/xep0308.py
  • nix run path:./vms#client (plain ./vms copied the 3.5 GB repo into the
    store on every commit), keyring crash recovery, screensaver off

Report: the app under test also shows its runs against peers in its own
matrix column; runs from VMs carry driven = "vm" (🤖 in matrix.md, 🖥 in
AI-RESULTS.md).

104 commits, runs one per commit (vm:), harness vms:.

🤖 Generated with Claude Code

The VM harness grown for real interop runs, and the 89 runs it recorded on 2026-11-08 (anotherim-js 0.8.0-rc9 / libtacky v1.2.3 against Gajim, Dino and Kaidan on prosody, ejabberd and anotherim-server; 77 pass, 12 fail). Bugs found: #9. Harness (vms/): - several VMs per driver run (`--peer`, nodes machine/peer/peer2, `boot_all`), disks per client and server so servers run in parallel - `fake_av`/`hears` (PipeWire null sink, sine on a fake mic, v4l2loopback camera, Goertzel), `net_down`/`netem`/`freeze` (+ `sync_clock`), `expect(text=, log=)`, `wait_notify` (dunst + dbus-monitor) - chat helpers per client (`open_chat`, `send`, `wait_message`) and `vm.py --record` writing run dirs; examples/xep0308.py - `nix run path:./vms#client` (plain ./vms copied the 3.5 GB repo into the store on every commit), keyring crash recovery, screensaver off Report: the app under test also shows its runs against peers in its own matrix column; runs from VMs carry `driven = "vm"` (🤖 in matrix.md, 🖥 in AI-RESULTS.md). 104 commits, runs one per commit (`vm:`), harness `vms:`. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
asxpi added 104 commits 2026-11-08 19:27:23 +00:00
The XEP-0308 runs needed two driver processes talking through files,
and nothing could check calls, reconnects or notifications.

- One driver, up to three VMs: nodes machine, peer and peer2, added
  with vm.py --peer CLIENT:JID and booted together by boot_all(). Each
  VM has its own out dir (<out>/<client>/), password dir and disk.
- Disks are per client and server (<client>-<server>.qcow2), so runs
  on different servers, with their own vm-* accounts, can go in
  parallel.
- fake_av(freq): PipeWire null speaker, a sine on a remapped fake mic
  and a test pattern on a v4l2loopback camera; hears(freq) records the
  speaker and checks the tone (Goertzel).
- net_down/net_up (QEMU set_link), netem (tc on eth0) and freeze (QEMU
  stop/cont).
- expect(text=, log=): waits for the screen and the log together and
  says which side missed.
- wait_notify(): dunst plus dbus-monitor logging every Notify call.

harness-test.py checks all of it with Gajim and Dino on prosody; it
passes (sign-in 29 s, message and notification 9 s, netem RTT 300 ms,
own tone 0.97-0.98 against 0.00 for the other).

Assisted-by: Claude:opus-5-5
The XEP-0308 VM runs were driven by ad-hoc snippets of absolute clicks,
which broke whenever a list's order or a window's layout differed.
ClientVM gains open_chat(who) and send(text), per client like the
_sign_in_<client> methods: keyboard where the client has it
(anotherim-js Esc, Ctrl+F, Alt+1; Gajim Ctrl+N; Dino Ctrl+T), clicks
relative to the main window elsewhere (Kaidan's list search, the
composers). send() waits for the message in the client's stanza log,
read on the host across lines, as Dino and Kaidan log a stanza over
several; wait_message() checks the receiving side. A message counts when
it has an id and a body or an OMEMO payload, so chat states and Kaidan's
OMEMO 2 key messages do not.

focus() and click_in() fall back to the window class: Kaidan's main
window has no name xdotool matches, and the old --sync name search hung
the driver forever. The guest cleans /tmp on boot: it is on the kept
disk, and a Kaidan killed at shutdown left its single-instance socket
there, so the next one only poked the dead socket and exited.

Assisted-by: Claude:opus-5-5
Last night's XEP-0308 VM runs became run folders through a one-off
script that wrote session.json and copied logs by hand, with versions
typed in. vm.py --record FEATURE (from the repo root) now does that: it
makes runs/<UTC start>-<FEATURE>-<peer>-<server>/ with the session.json
that interop start writes, under test anotherim-desktop at the version in
the AppImage's name (or --under-test-version), the peer's version from
the VM's own Nix package, the server's from servers.toml. Afterwards it
copies aim's logs in as client.log and client-debug.log, the peer's as
peer-<client>.log, and the script's shot-*.png. The guests log in UTC,
so tz_offset is 0 whatever the host's zone.

It does not finish the run: the agent looks at the screenshots, judges,
and runs interop finish (which fetches server.log, which check needs),
then marks the run driven = "vm"; the README gives the exact sequence,
and vm.py prints it with the run's name. The README also notes that
Dino cannot sign in to anotherim-server.

Assisted-by: Claude:opus-5-5
Last night's XEP-0308 check took a person-sized session of ad-hoc
snippets per client. examples/xep0308.py does it as one script on
open_chat() and send(): anotherim-js and a peer each send a message and
correct it, the logs show each <replace/> leave and arrive, and the
screenshots are named for the run folder, so with vm.py --record it
yields a run ready for the agent to judge and finish. It ran once against
Prosody with Dino (both ways, about four minutes) and the finished run
passed interop check; the correction steps for Gajim and Kaidan come
from the earlier hand runs and are not run through this script yet.

wait_message() takes has=, a regex the message must contain, to tell a
correction from the message it corrects.

Assisted-by: Claude:opus-5-5
Re-record of the correction run with rc9 (develop 142ee094, libtacky
v1.2.3), both directions in GUI VMs. AnotherIM's OMEMO 0.3 correction
carries <replace id='1794139278845460'/> outside the envelope; Gajim
logs that correction_id and shows one edited message 'two'. Gajim's
plaintext correction (Ctrl+Up) names the original id and AnotherIM
shows one bubble 'four' marked edited.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) as vm-aim and
Gajim 2.6.0 as vm-gajim, each in its own vms/ VM on ejabberd 26.09,
driven by vms/examples/xep0308.py.

AIM corrected an OMEMO 0.3 message with Up: new id, <replace/> naming
the original outside the envelope, and Gajim shows one message with
the corrected text and its pencil mark. Gajim corrected a plaintext
message with Ctrl+Up: AnotherIM shows one bubble with the new text,
'edited'. Four screenshots of both ends.

Assisted-by: Claude:opus-5-5
Re-record with rc9 (develop 142ee094, libtacky v1.2.3) in GUI VMs,
both ways under OMEMO 0.3. AnotherIM's correction carries <replace
id='1794139574658675'/> outside the envelope and Dino shows one
message 'two (edited)'; Dino's inline-editor correction names the
original id and AnotherIM shows one bubble 'four' marked edited.

Assisted-by: Claude:opus-5-5
anotherim-js started while gnome-keyring was still coming up could not
read its data key and stopped at "Your data cannot be opened" (seen on
anotherim-server with two VMs booting at once). Ping
org.freedesktop.secrets on the session bus first, which also activates
the daemon.

Assisted-by: Claude:opus-5-5
Corrections both ways in the GUI clients in VMs (rc9, libtacky v1.2.3).
AnotherIM's Up-edit went out as <replace id='1794139674270672'/> next
to the OMEMO 0.3 envelope and Gajim 2.6 shows one bubble "two" with its
pencil mark; Gajim's Ctrl+Up edit names the original's origin-id and
AnotherIM shows one bubble "four", marked edited.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Dino 0.5.1
in vms/ VMs on ejabberd 26.09, driven by vms/examples/xep0308.py.

Both ways over OMEMO 0.3: each correction carries <replace/> with the
original's id outside the envelope, and each side shows one message
with the corrected text and an edited mark. Dino's screen shows
'four' highlighted, which the harness's OCR missed; the screenshot
has it.

Assisted-by: Claude:opus-5-5
rc9 talks OMEMO 2 to Kaidan in both directions, so AnotherIM's own
correction carries <replace/> inside the SCE envelope too and the
aim-corrects step timed out although both screens showed 'two'. Wait
for the message alone with Kaidan, as peer-corrects already does.

Assisted-by: Claude:opus-5-5
Corrections both ways in the GUI clients in VMs (rc9, libtacky v1.2.3),
both under OMEMO 2, so the <replace/> is inside the envelope and the
screens judge: Kaidan shows one bubble "two" with its pencil mark,
AnotherIM one bubble "four" marked edited.

Assisted-by: Claude:opus-5-5
OMEMO 0.3 both ways in the GUI clients in VMs (rc9, libtacky v1.2.3):
AnotherIM's axolotl message carries a key for Gajim's device and Gajim
shows the text with its shield; Gajim's (OMEMO switched on in its lock
menu) arrives with a key for AnotherIM's device and shows decrypted.
Gajim lists AnotherIM's device as blind-trusted; AnotherIM offers the
new Gajim device for review.

Assisted-by: Claude:opus-5-5
Re-record with rc9 (develop 142ee094, libtacky v1.2.3) in GUI VMs.
Both ways OMEMO 2, so <replace/> sits inside the SCE envelope and the
screens decide: Kaidan shows AnotherIM's 'one' as one bubble 'two'
with its pencil, AnotherIM shows Kaidan's menu-edited 'three' as one
bubble 'four' marked edited. Notes an undecryptable pair of body-less
OMEMO 2 stanzas from an aborted attempt just before.

Assisted-by: Claude:opus-5-5
OMEMO 0.3 both ways in GUI VMs with rc9. Gajim's encryption was
switched on from its composer menu; AnotherIM's axolotl message is
decrypted by Gajim (its log and screen), and Gajim's comes out as an
encrypted bubble in AnotherIM. AnotherIM's review banner opens the
three Gajim devices with OMEMO 0.3 fingerprints, all undecided.

Assisted-by: Claude:opus-5-5
OMEMO 2 both ways in the GUI clients in VMs (rc9, libtacky v1.2.3):
AnotherIM's urn:xmpp:omemo:2 message carries a key for Kaidan's device
1156 and Kaidan shows the text; Kaidan's arrives with a key for
AnotherIM's device and shows decrypted. AnotherIM's review panel lists
Kaidan's device (OMEMO 0.9) as undecided. Both chats also show one or
two undecryptable messages from the session's first contact, noted.

Assisted-by: Claude:opus-5-5
OMEMO 0.3 both ways in GUI VMs with rc9. Each side's axolotl message
carries keys for the other's devices and shows decrypted on the other
screen. AnotherIM flags Dino's device as older OMEMO 0.3 and its review
pane lists both Dino devices with fingerprints, undecided.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Kaidan
0.16.0 in vms/ VMs on ejabberd 26.09. Both ways OMEMO 2, so the
<replace/> is inside the envelope and the verdict rests on the
screens: each side shows one bubble with the corrected text and its
edited mark.

Flaky before this run: twice Kaidan kept 'three' in its composer and
sent nothing to vm-aim; AnotherIM now shows three unreadable OMEMO
messages from those attempts, and Kaidan an undecryptable bubble for
AnotherIM's empty OMEMO 2 key exchange. Noted in the run, followed up
under XEP-0384.

Assisted-by: Claude:opus-5-5
OMEMO 2 both ways in GUI VMs with rc9: each side's urn:xmpp:omemo:2
message shows decrypted on the other screen, and AnotherIM's review
pane lists Kaidan's three devices with fingerprints. Also records that
AnotherIM's undecryptable-message notice in this chat grew from 2 to 3
during the earlier XEP-0308 run, correcting that run's note.

Assisted-by: Claude:opus-5-5
Receipts and read markers both ways in the GUI clients in VMs (rc9,
libtacky v1.2.3): Gajim's <received/> names AnotherIM's message id and
AnotherIM shows grey double ticks, then the read eye after Gajim opens
the chat; AnotherIM answers Gajim's request once to its full JID and
sends <displayed/> only after opening the chat. Both desktops showed a
notification with sender and text.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Gajim 2.6.0
in vms/ VMs on ejabberd 26.09, Gajim's chat set to OMEMO with its lock
menu.

A message each way decrypts on the other side, both over OMEMO 0.3
(eu.siacs.conversations.axolotl, the only version Gajim offers), with
the encrypted stanza leaving one log and arriving in the other. The
fingerprint AnotherIM shows for Gajim's device matches Gajim's own;
Gajim lists AnotherIM's device under blind trust, AnotherIM marks
Gajim's undecided and offers a review.

Assisted-by: Claude:opus-5-5
Receipts both ways in GUI VMs with rc9 under OMEMO 0.3: Gajim returns
<received/> for AnotherIM's message before its chat is open (grey
double ticks, Gajim's desktop notification logged), and AnotherIM
returns <received/> for Gajim's while its chat is closed. The displayed
markers that follow on opening are recorded too (blue ticks, Gajim's
read marker).

Assisted-by: Claude:opus-5-5
Read markers both ways in the GUI clients in VMs (rc9, libtacky
v1.2.3): no <displayed/> while the reader's window is minimized or its
chat closed, one <displayed/> naming the message's id after it opens
the chat. AnotherIM moves its read eye to the bubble Gajim marked;
Gajim 2.6 shows no read state on its own messages.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Dino 0.5.1
in vms/ VMs on ejabberd 26.09, OMEMO on by default in both.

A message each way decrypts on the other side over OMEMO 0.3, the
encrypted stanza leaving one log and arriving in the other. AnotherIM
flags Dino's device as the older OMEMO 0.3 and lists its fingerprint
as undecided, with a review prompt.

Assisted-by: Claude:opus-5-5
Displayed markers both ways in GUI VMs with rc9: none while the chat
is closed, then <displayed/> once the reader opens it. AnotherIM turns
its ticks blue with a seen count for Gajim's marker; Gajim draws
AnotherIM's read-marker avatar under its message.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Kaidan
0.16.0 in vms/ VMs on ejabberd 26.09.

A message each way decrypts on the other side over OMEMO 2, the
encrypted stanza leaving one log and arriving in the other; AnotherIM
lists Kaidan's device fingerprint as undecided, Kaidan reports
unverified contact devices.

Not judged, noted in the run: AnotherIM shows four unreadable OMEMO
messages from Kaidan left over from earlier attempts in which Kaidan
held a message without sending it, and Kaidan an undecryptable bubble
for AnotherIM's empty OMEMO 2 key exchange.

Assisted-by: Claude:opus-5-5
Receipts and read markers both ways in the GUI clients in VMs (rc9,
libtacky v1.2.3). Kaidan's receipt and marker travel inside OMEMO 2,
so AnotherIM's ticks (grey double, then the read eye after Kaidan opens
the chat) are the proof; AnotherIM's own receipt and <displayed/> go in
the clear and Kaidan shows its tick and "has read up to this point".

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Gajim 2.6.0
in vms/ VMs on ejabberd 26.09, each message sent while the other side
had the chat closed.

Each side requests a receipt and the other answers <received/> with
the message's id: AnotherIM's bubble goes to grey double ticks on
delivery, blue once Gajim opens the chat. Gajim notified the incoming
message on the desktop; AnotherIM, focused, showed an unread badge and
no desktop notification.

Assisted-by: Claude:opus-5-5
Read markers both ways in the GUI clients in VMs (rc9, libtacky
v1.2.3). Kaidan's <displayed/> is inside OMEMO 2: AnotherIM's read eye
moves to the new bubble only after Kaidan opens the chat. AnotherIM's
<displayed/> goes in the clear once its chat is opened, and Kaidan
shows "vm-aim has read up to this point".

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Gajim 2.6.0
in vms/ VMs on ejabberd 26.09, each message sent while the other side
had the chat closed.

Each side marks its message markable, and the <displayed/> naming it
comes back only once the other side opens the chat: AnotherIM's ticks
go from grey to blue with a seen-by mark, Gajim puts AnotherIM's read
avatar under its message.

Assisted-by: Claude:opus-5-5
A thumbs-up reaction each way in the GUI clients in VMs (rc9, libtacky
v1.2.3): AnotherIM's menu quick row and Gajim's hover bar each send one
<reactions/> naming the other's message id with a <store/> hint, and
both screens show the "👍 1" pill on the right bubble.

Assisted-by: Claude:opus-5-5
Receipts both ways in GUI VMs with rc9 under OMEMO 0.3, each sent
while the receiver had the chat closed: AnotherIM shows grey double
ticks for Dino's receipt, Dino a single tick for AnotherIM's. The
displayed markers after opening the chat are recorded alongside.

Assisted-by: Claude:opus-5-5
Once in a few starts rc9 came up on "Your messages are locked ... This
device cannot open your messages right now" instead of signing in, so
wait_main sat out its 180 s for a roster that never came. While the
roster is missing, look for that dialog every 10 s and press its
Unlock, which the dialog itself suggests ("try again in a moment").

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Dino 0.5.1
in vms/ VMs on ejabberd 26.09.

Each side requests a receipt and the other answers <received/> with
the message's id; AnotherIM's bubble and Dino's check marks show
delivery, then read. Dino reopens its last chat at start, so its read
marker came at once and it raised no notification; AnotherIM, focused,
showed an unread badge and no desktop notification.

Assisted-by: Claude:opus-5-5
Displayed markers both ways in GUI VMs with rc9: sent only once the
reader opens the chat (Dino parked elsewhere first, as it reopens the
last chat). AnotherIM shows blue ticks and a seen count for Dino's
marker; Dino turns its tick into the double 'Read' tick for AnotherIM's.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Dino 0.5.1
in vms/ VMs on ejabberd 26.09.

Both send <markable/> messages and answer <displayed/> with the
other's id: AnotherIM only once its chat is opened, Dino at once since
it keeps its last chat open from start. AnotherIM's ticks turn blue
with a seen-by mark; Dino's mark turns to read.

Assisted-by: Claude:opus-5-5
Receipts both ways in GUI VMs with rc9 under OMEMO 2. Kaidan puts its
receipt inside the OMEMO 2 envelope, so aim's log holds no plaintext
<received/>, but AnotherIM decrypts it and shows grey double ticks;
AnotherIM's plaintext receipt gives Kaidan its tick. Kaidan's desktop
notification and the displayed markers are recorded too.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Kaidan
0.16.0 in vms/ VMs on ejabberd 26.09, OMEMO 2 both ways.

Kaidan's receipt travels inside the OMEMO 2 envelope (a body-less
encrypted message), AnotherIM's in clear; each side shows delivery
(AnotherIM's grey double ticks, Kaidan's check) and then read. Kaidan
notified the incoming message on the desktop; AnotherIM, focused,
showed an unread badge only.

Assisted-by: Claude:opus-5-5
Displayed markers both ways in GUI VMs with rc9 (OMEMO 2 bodies,
plaintext markers): none until the reader opens the chat, then
AnotherIM shows blue ticks and a seen count, and Kaidan its 'has read
up to this point' line.

Assisted-by: Claude:opus-5-5
A thumbs-up reaction each way in GUI VMs with rc9: AnotherIM's from
its message menu's quick bar, Gajim's from its hover bar. Each
<reactions/> names the other's message id, and both screens show 👍 1
under the right message.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Kaidan
0.16.0 in vms/ VMs on ejabberd 26.09, OMEMO 2 both ways.

Each side's <displayed/> names the other's message id and comes only
once the chat is opened: AnotherIM's bubble gains its seen-by mark,
Kaidan shows "vm-aim has read up to this point".

Assisted-by: Claude:opus-5-5
A reply each way in the GUI clients in VMs (rc9, libtacky v1.2.3):
AnotherIM's reply names Gajim's origin-id with a "> " fallback range,
and Gajim shows it under a quote card; Gajim's names AnotherIM's id and
AnotherIM shows it under a "You" quote with the fallback stripped.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Gajim 2.6.0
in vms/ VMs on ejabberd 26.09.

Each side puts a thumbs up on the other's message from its own UI
(AnotherIM's menu quick bar, Gajim's hover bar): <reactions/> naming
the message's id goes out and arrives, and both sides show 👍1 under
the right message.

Assisted-by: Claude:opus-5-5
A thumbs-up reaction each way in GUI VMs with rc9: AnotherIM's from
its menu's quick bar, Dino's from its emoji chooser. Each <reactions/>
names the other's message id and both screens show 👍 1 under it.

Assisted-by: Claude:opus-5-5
A retraction each way in the GUI clients in VMs (rc9, libtacky
v1.2.3): AnotherIM's Delete for everyone sends <retract/> naming its
own id with a fallback body, and Gajim greys the row out as retracted;
Gajim's Retract names its origin-id and AnotherIM shows "Message
retracted" with no extra row from the fallback text.

Assisted-by: Claude:opus-5-5
A reply each way in GUI VMs with rc9: each carries <reply id to/>
naming the other's message plus an XEP-0428 fallback range over the
quoted lines. Both clients draw a quote block for the replied-to
message and strip the fallback text from the body.

Assisted-by: Claude:opus-5-5
AnotherIM's network unplugged for 30 s in its VM (rc9, libtacky v1.2.3)
while Gajim sent a message: the stream survived, acks resumed, and the
message arrived once after the link came back and shows once. Stream
resumption itself was not needed in this run.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Gajim 2.6.0
in vms/ VMs on ejabberd 26.09, OMEMO 0.3.

Each side replies to the other's message from its own UI: <reply/>
naming the message's id and its author, with a XEP-0428 fallback,
travels next to the encrypted body, and both sides draw the quote of
the right message above the reply.

Assisted-by: Claude:opus-5-5
./vms resolves to a git flake, so every nix run of it copies the whole
interop repo, LFS run logs included (3.5 GB), into /nix/store, and again
after each commit. Three agents doing that filled the root disk. path:./vms
copies only vms/. The record section also said to work inside nix
develop, which copies the repo the same way; use nix shell instead.

Assisted-by: Claude:opus-5-5
A run "anotherim-desktop vs Gajim" was filed only under its peer's
column, so anotherim-desktop's own column stayed untested although
every VM run tested it.

An under-test column now also shows that app's runs on the server
against any peer when it has no run of its own there: the worst
verdict, xN how many peers gave it, linked to the newest such run;
people's runs before agents' (marked with a robot).

Assisted-by: Claude:opus-5-5
A thumbs-up reaction each way in GUI VMs with rc9 on an OMEMO 2 chat:
AnotherIM from its message menu quick bar, Kaidan from its message icon
bar and emoji search. Both screens show 👍 1 under the right message.

Kaidan encrypts its reaction; AnotherIM sends its <reactions/> in the
clear although the chat is OMEMO 2. Kaidan takes it fine, so this is a
pass, but the reaction is visible to the server.

Assisted-by: Claude:opus-5-5
AnotherIM's VM paused 330 s (rc9, libtacky v1.2.3) while Gajim sent
three messages: after resuming, the stream was resumed and the MAM
catch-up returned the same three again, and AnotherIM shows each once
in order with one receipt each. A resume refused after the server's
timeout was not reached; the server kept the session.

Assisted-by: Claude:opus-5-5
A reply each way in GUI VMs with rc9 on an OMEMO 0.3 chat: AnotherIM
from its message menu, Dino from its hover bar. Both <reply/> stanzas
name the right message id with an XEP-0428 fallback, and each screen
quotes the replied-to message.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Gajim 2.6.0
in vms/ VMs on ejabberd 26.09.

Each side deletes its own message for everyone from its UI (AnotherIM:
menu Delete and confirm; Gajim: Retract... and confirm): a
message-retract:1 <retract/> naming the message's id goes out and
arrives, and both sides replace that message with a retracted notice.

Assisted-by: Claude:opus-5-5
300 ms delay and 5 % loss on AnotherIM's link in its VM (rc9, libtacky
v1.2.3) while five messages went each way: every message arrived once
and in order on both screens, each of AnotherIM's got Gajim's receipt,
and the stream never dropped.

Assisted-by: Claude:opus-5-5
A reply each way in GUI VMs with rc9 on an OMEMO 2 chat: AnotherIM
from its message menu, Kaidan from its message icon bar. Both put the
<reply/> inside the OMEMO 2 envelope, so the verdict rests on the
screens, which quote the right message each way.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Dino 0.5.1
in vms/ VMs on ejabberd 26.09.

Each side puts a thumbs up on the other's message (AnotherIM's menu
quick bar, Dino's emoji chooser): <reactions/> naming the message's id
crosses, and both sides show 👍1 under the right message.

Assisted-by: Claude:opus-5-5
A pasted picture each way in the GUI clients in VMs (rc9, libtacky
v1.2.3): AnotherIM's upload goes out as an OMEMO aesgcm:// link and
Gajim shows it inline; Gajim's arrives as aesgcm (OMEMO on) and as a
plain OOB https link (OMEMO off), and AnotherIM shows both inline, the
plain one marked not encrypted.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Dino 0.5.1
in vms/ VMs on ejabberd 26.09, OMEMO 0.3.

Each side replies to the other's message from its UI: <reply/> with
the quoted message's id and a fallback travels next to the encrypted
body, and both draw the quote above the reply. Aside: Dino names its
own full JID in the reply's 'to' while quoting AnotherIM; AnotherIM
still attributes the quote by id.

Assisted-by: Claude:opus-5-5
Two AnotherIM sessions on one account plus Gajim, three VMs (rc9,
libtacky v1.2.3): each AnotherIM gets a <sent/> carbon of the other's
message to Gajim and shows it as its own, decrypted; Gajim's message to
the bare JID shows once in both. The second session's XML log is in the
run as client-2.log.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Kaidan
0.16.0 in vms/ VMs on ejabberd 26.09, OMEMO 2.

AnotherIM's thumbs up on Kaidan's message and Kaidan's 😀 on
AnotherIM's show on both sides under the right message. AnotherIM
sends its <reactions/> in clear even in this OMEMO chat; Kaidan's
travels inside the OMEMO 2 envelope.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Kaidan
0.16.0 in vms/ VMs on ejabberd 26.09, OMEMO 2.

Each side replies to the other's message from its UI and both draw
the quote of the right message above the reply. Under OMEMO 2 the
<reply/> is inside the envelope, so this rests on the screenshots.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Gajim 2.6.0
in vms/ VMs on ejabberd 26.09.

AnotherIM's NIC was unplugged for 30 s while Gajim sent it a message.
Once the link was back the message arrived, AnotherIM resumed its
stream inline in SASL2 (<resumed h='41'/>, no new bind), and it shows
the message once; later MAM/carbon copies of the same id were
de-duplicated.

Assisted-by: Claude:opus-5-5
A thumbs-up reaction each way in the GUI clients in VMs (rc9, libtacky
v1.2.3): Kaidan shows AnotherIM's reaction under its message, and
AnotherIM decrypts Kaidan's OMEMO 2 reaction and shows the pill.
AnotherIM's own <reactions/> go in the clear, Kaidan's inside OMEMO 2.

Assisted-by: Claude:opus-5-5
AnotherIM creates and joins a run-unique room from its Join room sheet
(201 + 110, owner), Gajim joins it, and a groupchat line each way shows
on both screens with the right roles. Noted: AnotherIM sends no
instant-room confirmation after the 201; ejabberd keeps the room open,
a server that locks new rooms would not.

Assisted-by: Claude:opus-5-5
A retraction each way in GUI VMs with rc9 on an OMEMO 0.3 chat, both
in the clear: AnotherIM from its message menu (Delete, for everyone),
Gajim from its hover bar menu (Retract...). Each <retract/> names the
sender's own message id with a fallback body, and the other side
replaces the message with its retracted line.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Gajim 2.6.0
in vms/ VMs on ejabberd 26.09.

AnotherIM's VM was paused and unplugged for 345 s, past the server's
300 s resume window, while Gajim sent two messages. After resuming it
shows both once, in order, and its MAM catch-up ran without
duplicates. Caveat in the run: ejabberd never noticed the silent
socket, so the stream was resumed and the messages came live; a
catch-up after a dropped session is not shown here.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Gajim 2.6.0
in vms/ VMs on ejabberd 26.09, AnotherIM's link at 300 ms delay and
5 % loss (tc netem).

A burst of five messages each way: all ten arrived once and in order,
and every one got its delivery receipt and read marker back, naming
its id; both screens show the bursts in order with ticks.

Assisted-by: Claude:opus-5-5
A reply each way under OMEMO 2 in the GUI clients in VMs (rc9, libtacky
v1.2.3): the reply elements are inside the envelope, and each side
shows the other's reply under a quote of the right message.

Assisted-by: Claude:opus-5-5
Retraction does not work either way on an OMEMO 2 chat with rc9.

AnotherIM sends its <retract/> in the clear although the chat is
OMEMO 2; Kaidan shows the fallback body as a new unencrypted message
and keeps the original. Kaidan sends its retraction inside OMEMO 2;
AnotherIM keeps the message and its "can't be read on this device"
count goes up by one, as it did in a first attempt.

Kaidan writes every message to AnotherIM's device as the same
unconfirmed key exchange (pk_id 39), while AnotherIM answers on an
older established session; most of those still decrypt.

Assisted-by: Claude:opus-5-5
990eb22 committed the run without its run.toml: the fail verdict
needed a --bug, so finish refused. Add it and the report.

Retraction does not work either way on an OMEMO 2 chat with rc9.
AnotherIM sends its <retract/> in the clear although the chat is
OMEMO 2; Kaidan shows the fallback body as a new unencrypted message
and keeps the original. Kaidan sends its retraction inside OMEMO 2;
AnotherIM keeps the message and its "can't be read on this device"
count goes up by one, as it did in a first attempt.

Kaidan writes every message to AnotherIM's device as the same
unconfirmed key exchange (pk_id 39), while AnotherIM answers on an
older established session; most of those still decrypt.

Assisted-by: Claude:opus-5-5
AnotherIM's link goes down for about 40 s while Gajim sends. The TCP
stream survives, so there is no resumption; once the link is back SM
acks flow and Gajim's message arrives once, live, shown once.

Assisted-by: Claude:opus-5-5
AnotherIM's retraction, sent in the clear in an OMEMO 2 chat, reaches
Kaidan but is not applied: the message stays and the fallback body
appears as a new unencrypted row, three times in a row. Gajim applies
the same retraction. Kaidan's own trash only deletes locally, so the
other direction was not exercised.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Dino 0.5.1
in vms/ VMs on ejabberd 26.09, with fake microphones (440 and 660 Hz)
and speakers the harness records.

An audio call started from AnotherIM's header rang Dino (XEP-0353),
was accepted, set up over ICE and carried each side's tone to the
other; Hang up ended it on both. A video call negotiated audio and
video both ways, but AnotherIM played no audio from Dino in it while
Dino heard AnotherIM; the cause is not established. Video frames are
black in both windows' screenshots and not judged.

Assisted-by: Claude:opus-5-5
A pasted picture from AnotherIM to Kaidan under OMEMO 2 (rc9, libtacky
v1.2.3): the upload and the message go through, but Kaidan refuses the
download with "Transfer failed: Checksum does not match" and shows no
picture. Gajim, which checks no hash, shows the same kind of upload.
Kaidan's own upload was not driven (paste does not attach there).

Assisted-by: Claude:opus-5-5
Same script and VMs as the run before, to check the video-call result.
The audio call again carries both tones and hangs up cleanly; in the
video call AnotherIM again plays nothing from Dino while Dino hears
AnotherIM, so the silence reproduces.

Assisted-by: Claude:opus-5-5
The checksum Kaidan rejects is right: AnotherIM's stored message names
sha-256 e7dtPevD…6a8= for the file, and the uploaded object, fetched
and decrypted with the key from its aesgcm fragment, is a 3444-byte PNG
with exactly that hash. AnotherIM announces the plaintext's hash, as
file metadata should; Kaidan checks it against something else, likely
the encrypted download. Cause moved from libtacky to peer.

Assisted-by: Claude:opus-5-5
AnotherIM's VM is paused for 700 s while Gajim sends twice. Prosody
still held the session, so AnotherIM resumes and also catches up from
MAM; each message arrives twice (SM replay and MAM) and is shown once,
in order.

AnotherIM shows the replayed message at its receive time although the
stanza carries an XEP-0203 delay stamp; noted in the run.

Assisted-by: Claude:opus-5-5
QEMU stop/cont leaves the guest's wall clock behind by the whole pause;
a laptop resuming from suspend would catch up from its RTC. After a
700 s freeze AnotherIM's clock was 12 minutes slow and it stamped
received messages with that time, which made the run look like a
timestamp bug. Add sync_clock() (date -s from the host) and call it
from freeze(); scripts that stop/cont by hand should call it too.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Kaidan
0.16.0 in vms/ VMs on ejabberd 26.09.

Kaidan rang (desktop notification, accepted through its default
action) and answered both an audio and a video call, but its
session-accept has no <rtcp-mux/>, and AnotherIM's WebRTC engine
rejects that answer ("RTCP-MUX is not enabled when it is required"):
the audio call stays ringing, the video call ends in failed-transport,
no audio either way. rtcp-mux is optional in XEP-0167, so AnotherIM
should accept such an answer.

Assisted-by: Claude:opus-5-5
AnotherIM's link delayed 300 ms with 5 % loss; a burst of five each
way. Every one of AnotherIM's messages gets its receipt and double
tick, and both screens show both bursts in order, none lost or doubled.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Gajim 2.6.0
in vms/ VMs on ejabberd 26.09, OMEMO 0.3 chat.

A picture each way from each client's attach UI: an encrypted upload
(slot size = file + GCM tag) to the server's HTTP upload service and
an aesgcm:// link inside the OMEMO message; both sides show both
pictures inline. Aside: AnotherIM adds a SIMS reference with file
name, type and size in clear next to the encrypted payload.

Assisted-by: Claude:opus-5-5
AnotherIM creates a room (201) but never sends the muc#owner instant-
room submit. Prosody keeps the room locked, as XEP-0045 10.1 says, so
Gajim's join is refused with item-not-found. ejabberd and anotherim-
server open such rooms anyway, which hid this there.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Gajim 2.6.0
in vms/ VMs on ejabberd 26.09, Gajim's chat set to no encryption.

Gajim's plaintext upload (https body plus jabber❌oob) shows inline
in AnotherIM, marked not encrypted. The harness could not switch
AnotherIM's own chat to no encryption (the long press on its lock was
taken as a tap), so AnotherIM's picture went encrypted again and
showed in Gajim; its plaintext upload is not covered here.

Assisted-by: Claude:opus-5-5
About one Gajim boot in five, the daemon pam_gnome_keyring started at
login aborted (assertion 'client' failed, SIGABRT) while D-Bus
activated a second one, and Gajim came up asking for its password.
The harness keyring has no password, so the PAM start buys nothing:
drop pam_gnome_keyring from login (lightdm-autologin includes lightdm,
which includes login) and let D-Bus start one daemon on demand.

Checked on prosody: harness-test.py with Gajim and Dino passes (both
sign in from the keyring), and AnotherIM and Kaidan start signed in,
the Secret Service answering, one daemon per guest.

harness-test.py now also checks that the guest clock matches the
host's after freeze() (sync_clock): off by 0.0 s.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Dino 0.5.1
in vms/ VMs on ejabberd 26.09, OMEMO 0.3 chat.

A picture each way from each client's attach UI: an encrypted upload
to the server's HTTP upload service and the link inside the OMEMO
message; both sides show both pictures inline. Neither side used SFS.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Kaidan
0.16.0 in vms/ VMs on ejabberd 26.09, OMEMO 2 chat.

AnotherIM's encrypted upload reaches Kaidan, which downloads it and
rejects it with "Checksum does not match" (reproduced on two
uploads), so no picture shows; Gajim and Dino display the same kind
of upload. Kaidan's picture arrives in AnotherIM as a file with a
random-looking name and a Download link rather than inline. The hash
metadata is inside the OMEMO 2 envelope, so the cause rests on the
screens; libtacky is suspected.

Assisted-by: Claude:opus-5-5
Three VMs: AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3)
twice as vm-aim, one of them on a fresh disk signed in through its own
UI, and Gajim 2.6.0 as vm-gajim, on ejabberd 26.09.

Both AnotherIM instances enable carbons; a message each sends to Gajim
reaches the other as a <sent/> carbon and shows there as its own
outgoing message. The second instance's XML log is in the run as
client-2.log. Gajim -> vm-aim (received carbons) is not covered: a
leftover draft turned Gajim's send into an unknown command.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Gajim 2.6.0
in vms/ VMs on ejabberd 26.09.

AnotherIM created a fresh room through its Join room sheet (status 110
and 201, instant room), Gajim joined it through Start / Join Chat, and
a groupchat line each way shows on both sides, with both occupants
and their roles listed.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Gajim 2.6.0
in vms/ VMs on ejabberd 26.09, in a room AnotherIM created.

Joining published an autojoin bookmark to urn:xmpp:bookmarks:1; after
AnotherIM was quit and restarted it read its bookmarks and rejoined
the room unprompted, and Gajim saw vm-aim come back. Bookmarks are per
account, so this shows AnotherIM's own round trip with a peer in the
room, not cross-client sync.

Assisted-by: Claude:opus-5-5
X's screensaver blanked the screen after about ten idle minutes, so
the final screenshot of a 13-minute outage run came out black. Turn
the screensaver and DPMS off at boot (xset, now in the image).

Assisted-by: Claude:opus-5-5
AnotherIM's link is down for 780 s while Gajim sends twice. It
resumes, also catches up from MAM, and shows each message once at its
<delay> time, with the guest clock now correct.

The earlier run's note that AnotherIM ignores the delay stamp came
from the stale clock after a QEMU pause; mark it withdrawn there.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Dino 0.5.1
in vms/ VMs on ejabberd 26.09.

AnotherIM created a fresh room from its Join room sheet, Dino joined
it through Join Channel, and a groupchat line each way shows on both
sides, AnotherIM listing both occupants with their roles.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Dino 0.5.1
in vms/ VMs on ejabberd 26.09, in a room AnotherIM created and Dino
joined.

AnotherIM published an autojoin bookmark on joining and, after a quit
and restart, read it back and rejoined the room on its own. Bookmarks
are per account, so this is AnotherIM's own round trip with a peer in
the room.

Assisted-by: Claude:opus-5-5
As with Gajim: AnotherIM creates the room but sends no muc#owner
instant-room submit, Prosody keeps it locked, and Dino's join is
refused with item-not-found ("Room does not exist").

Assisted-by: Claude:opus-5-5
8448d94 took the login-time daemon out, but gnome-keyring 50 still
aborts now and then on a client's first Secret Service call, with
only the D-Bus-started daemon running (assertion 'client' failed in
gkd_secret_service_get_pkcs11_session). Gajim then asks for its
password and anotherim-js shows "Your data cannot be opened".

start() now waits 10 s after the client's window and, if the journal
shows a new gnome-keyring core dump, restarts the client: D-Bus starts
a fresh daemon on its next call. Checked by SIGABRTing the daemon 6 s
after anotherim-js starts: restarted, signed in.

Also: the keyring reset's pkill matched nothing (the daemon's name
is .gnome-keyring-, the Nix wrapper); match the full command line.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) calls Dino
0.5.1 in vms/ VMs on prosody, audio then video. Signalling completes
(propose, proceed, session-initiate, session-accept), but ICE never
connects: prosody answers the extdisco query with service-unavailable,
so both NATed VMs offer only host candidates, and both calls end in
connectivity-error ("Call failed to establish" on both screens).

The same script connects on ejabberd, which hands out STUN/TURN. Cause:
the server's missing STUN/TURN, not the clients.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) calls Kaidan
0.16 in vms/ VMs on prosody, audio then video, Kaidan accepting from
its notification. Reproduces the ejabberd result: Kaidan's
session-accept has no rtcp-mux, AnotherIM's WebRTC rejects the answer
("RTCP-MUX is not enabled when it is required"), the call never
connects and no audio flows. rtcp-mux is optional in XEP-0167, so the
cause stays with libtacky. Prosody's missing STUN/TURN would block ICE
here as well.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Gajim
2.6.0 in vms/ VMs on prosody exchange a picture each way in an OMEMO
0.3 chat. AnotherIM's slot request is file size + 16 (encrypted
upload) to mod_http_file_share, Gajim decrypts an aesgcm:// URL and
shows the colour bars inline; Gajim's gradient shows inline in
AnotherIM.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Gajim
2.6.0 in vms/ VMs on prosody, encryption switched off on both sides
(AnotherIM: padlock right click, No encryption, confirm). Each sends a
picture: AnotherIM uploads the plain 1216 bytes and sends the https
URL as body, OOB, SIMS and an XEP-0447 file-sharing element with hash
and thumbnail; Gajim shows it inline. Gajim's https+OOB picture shows
inline in AnotherIM, marked not encrypted. This also covers
AnotherIM's own plaintext upload, which the ejabberd run missed.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Dino 0.5.1
in vms/ VMs on prosody exchange a picture each way in an OMEMO 0.3
chat: AnotherIM's encrypted upload (slot size = file + 16) shows inline
in Dino, Dino's encrypted upload shows inline in AnotherIM. Same as on
ejabberd.

Assisted-by: Claude:opus-5-5
AnotherIM 0.8.0-rc9 (develop 142ee094, libtacky v1.2.3) and Kaidan
0.16 in vms/ VMs on prosody, OMEMO 2 chat, a picture each way. Both
ejabberd findings reproduce: Kaidan rejects AnotherIM's encrypted
upload with "Checksum does not match", and Kaidan's picture shows in
AnotherIM as a randomly named file with a Download link rather than
inline. The checksum was traced on anotherim-server to Kaidan (the
announced sha-256 matches the decrypted file), so cause is the peer.

Assisted-by: Claude:opus-5-5
Three vms/ VMs on prosody: AnotherIM 0.8.0-rc9 (develop 142ee094,
libtacky v1.2.3) twice as vm-aim and Gajim 2.6.0. Both AnotherIM
instances enable carbons. A plaintext message from the first reaches
the second as a sent carbon and shows as its own; an OMEMO message
from the second reaches the first as a sent carbon, is decrypted and
shows as its own; Gajim's message to the bare JID reaches both
directly; Gajim shows all three. The second AnotherIM's log is in the
run as client-2.log.

Assisted-by: Claude:opus-5-5
vm: XEP-0363 Kaidan on ejabberd: cause peer, not libtacky
Some checks failed
/ check (pull_request) Failing after 9m55s
596d76787a
The same failure on anotherim-server was traced to Kaidan: the file
decrypted from the upload has the sha-256 AnotherIM announced. Mark
the ejabberd run the same so the reports agree.

Assisted-by: Claude:opus-5-5
interop: let a run say why its feature markers are missing
All checks were successful
/ check (pull_request) Successful in 9m46s
d0f58bfe3d
CI failed six VM runs that pass but whose logs lack the feature's
markers. Under OMEMO 2 a correction or reply travels inside the SCE
envelope, so no log can show it and the agent judged it from both
screens; and the MUC passes on ejabberd and anotherim-server have no
muc#owner stanza because anotherim-js never sends one (#9 C35) while
those servers open the room anyway.

A run may now carry markers_missing = "<why>": check accepts the pass
and the reason stays in run.toml beside the verdict. Set it on the six
runs.

Assisted-by: Claude:opus-5-5
asxpi merged commit 67e99ae73a into main 2026-11-08 19:54:27 +00:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
another-im/interop!10
No description provided.