another-im/interop
Watch
9
0
Fork
You've already forked interop
 0
Code Issues 5 Pull requests 1 Releases 7 Activity Actions

vms: consolidate the run agents' scripts into the harness #14

Merged
asxpi merged 9 commits from vms/consolidate into main 2026-11-09 08:26:52 +00:00
Conversation 0 Commits 9 Files changed 42 +1935 -1464
asxpi commented 2026-11-09 05:31:06 +00:00
Member
Copy link

Turns the run agents' throwaway scripts into the harness.

  • interopvm.py: OCR text clicking (find_text/click_text), find_color, host-side test pictures, Run (numbered shots, PASS/FAIL/SKIP), web fake A/V order automatic, RTP counters (web getStats → web-rtp.log, Dino G_MESSAGES_DEBUG=rtp), test-card reading.
  • vms/clients/{aim,web,gajim,dino,kaidan}.py: per-client actions (attach, react, reply, retract, correct, join_room, set_encryption, start/accept/hang up calls, close_chat), NotImplementedError where a client lacks the feature.
  • vms/examples/: one script per feature (0045+0402, 0167, 0184+0333, 0198, 0280, 0308, 0313, 0363, 0384, 0424, 0444, 0461, netem), replacing vms/scenarios/; vmrun.zsh runs them.
  • vms/RUN-BRIEF.md: the brief template for VM run agents, with a mandatory "improve the harness before reporting" phase; AGENTS.md and README updated.

Verified on ejabberd with --out (nothing recorded): every example against every peer that supports it works, except Kaidan calls (C36) and one Dino video call that reproduced #9 P3. harness-test.py passes.

🤖 Generated with Claude Code

Turns the run agents' throwaway scripts into the harness. - `interopvm.py`: OCR text clicking (`find_text`/`click_text`), `find_color`, host-side test pictures, `Run` (numbered shots, PASS/FAIL/SKIP), web fake A/V order automatic, RTP counters (web getStats → `web-rtp.log`, Dino `G_MESSAGES_DEBUG=rtp`), test-card reading. - `vms/clients/{aim,web,gajim,dino,kaidan}.py`: per-client actions (attach, react, reply, retract, correct, join_room, set_encryption, start/accept/hang up calls, close_chat), NotImplementedError where a client lacks the feature. - `vms/examples/`: one script per feature (0045+0402, 0167, 0184+0333, 0198, 0280, 0308, 0313, 0363, 0384, 0424, 0444, 0461, netem), replacing `vms/scenarios/`; `vmrun.zsh` runs them. - `vms/RUN-BRIEF.md`: the brief template for VM run agents, with a mandatory "improve the harness before reporting" phase; AGENTS.md and README updated. Verified on ejabberd with `--out` (nothing recorded): every example against every peer that supports it works, except Kaidan calls (C36) and one Dino video call that reproduced #9 P3. `harness-test.py` passes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) 
Work in progress of the consolidation of the run agents' helpers;
untested in a VM so far.

find_text()/click_text() (host tesseract 5 from the flake, words and
lines, a --psm 11 second pass) replace find_word()/click_word();
find_color() finds white-on-colour buttons (anotherim-js's green Answer,
red Hang up) with the flake's ImageMagick. make_png() writes test
pictures on the host, as the guest's pngenc fails. Per-client actions
dispatched like _sign_in_<client>: attach, react, reply, retract,
correct, join_room, set_encryption, start_call, accept_call, hang_up,
with NotImplementedError where a client lacks the feature. Plus
bubble(), stanza()/wait_stanza(), log_since(), close_chat(),
call_diag() and read_testcard(). The guest gets xclip for pasting.

Assisted-by: Claude:opus-5-5
The examples should need nothing but interopvm, so the step and
screenshot bookkeeping each agent had copied (common.py, runlib.py) is
now Run: the VMs from the driver's nodes, numbered shot-*.png, steps
that turn NotImplementedError into a SKIP and stop after a failed
signin, setup or prep, and the RESULTS summary.

For web, fake_av only worked before Chromium started, and an earlier
run's files stayed on the disk: boot() drops them, and fake_av on a
running web restarts it (stop(), restart() are methods now). send()
clears a leftover draft first (Gajim keeps them, and one turned the
next message into a /command) and presses Return once more when the
message does not leave (Kaidan). Gajim runs unbuffered, so its log no
longer lags its sends.

Gajim's hover bar has its quick reactions only sometimes, so react
goes through the add-reaction button and the GTK emoji chooser (as for
Dino), with the window clicked first: the bar shows only in the active
window. Checked on ejabberd, both ways.

To tell why a call is silent: interop-web records every peer
connection's getStats counters to /out/web-rtp.log while a call is up,
Dino runs with G_MESSAGES_DEBUG=rtp, and call_diag() collects those
and a PipeWire dump. aim's native engine reports no RTP stats.
vm.py exports the recorded feature (INTEROP_VM_RECORD), so one
example can serve two specs.

Assisted-by: Claude:opus-5-5
The run agents left four sets of scripts with the same steps and
different click positions (ejabberd's in vms/scenarios, prosody's,
anotherim-server's and the web's in scratchpads). The positions and
workarounds are in interopvm's per-client actions now; what is left is
one example per feature, parametrised by the peer client (--peer):
xep0167 (calls), xep0184 (+0333), xep0198, xep0280, xep0308, xep0313,
xep0045 (+0402), xep0363 (ENC=omemo|plain), xep0424, xep0444, xep0461
and netem. vms/scenarios goes.

vmrun.zsh runs the examples, passes --web for a web peer and no longer
needs a nix shell for tesseract (the flake has it). env.zsh no longer
defaults INTEROP_BRANCH to vms/harness-1-5, which was merged and must
not get pushes; commit.zsh asks for it.

Assisted-by: Claude:opus-5-5
Every agent that taught the harness a client's clicks edited the same
stretch of interopvm.py. Each client's sign-in, chat and action methods
(_sign_in_/_open_chat_/_composer_ and _<action>_<client>) now live in
its own mixin, vms/clients/{aim,web,gajim,dino,kaidan}.py, which
ClientVM inherits, so the dispatch by name and the API stay the same
(checked: the same attributes, each with the same source). Generic
driving stays in interopvm.py. The flake's lib ships the clients/
package beside interopvm.py.

It also holds the fixes from the verification runs so far: aim's lock
menu is found without Esc (which closes the chat), Gajim's close_chat
is not scripted (its Ctrl+W asks to quit Gajim), and xep0184, xep0424
and xep0461 check the right stanza or line.

Assisted-by: Claude:opus-5-5
Running every example once on ejabberd showed:
- Kaidan takes no pasted picture: attach() goes through its + menu's
  folder icon and file chooser, as the ejabberd agent's script did.
- aim's lock tooltip covers the encryption menu's first item (OMEMO):
  move the pointer off the lock before reading the menu. web's header
  has no window buttons, so its lock (and phone) sit 110 px further
  right.
- tesseract drops a line of aim's message menu now and then: look for
  Reply/Forward with --psm 11 too.
- xep0167: Dino rings inside the caller's chat, so the peer opens it;
  wait for aim's session-terminate, its log lags the click.
- xep0308: aim and web talk OMEMO 2, so <replace/> is in the envelope.

Assisted-by: Claude:opus-5-5
- Gajim's lock is the OMEMO shield's neighbour: with OMEMO on, the
  last icon opens an "Encryption (OMEMO)" window, not the menu. Close
  that window and try the lock left of it, and never press Esc in the
  chat itself (it closes the chat).
- Dino's file chooser now and then ignores the typed path: pick the
  file's row and Select.
- read_testcard() also reads the frame counter beside BL, so two reads
  show whether the picture moves.
- harness-test.py drives the message through open_chat/send and waits
  for it with wait_message: it no longer assumes prosody.
- xep0384 checks for a clear body on the sender's side; Gajim logs the
  stanzas it decrypted.

Assisted-by: Claude:opus-5-5
README.md lists Run, every ClientVM action and helper and every
example, one line each, and what each client lacks. AGENTS.md points to
examples/ and clients/ instead of the scenarios, keeps the click
positions per client as the second pass left them, and the results
table now covers all VM runs, anotherim-web included (regenerated from
the runs' run.toml).

Stale text fixed: Dino on anotherim-server signs in since #11; "Dino
video calls on ejabberd: no audio" is now the intermittent Dino ->
anotherim-js silence (#9 P3), and every call run before c137b7b had no
camera feed. Open questions noted, not fixed: aim's outgoing video
never shows (web's getStats sees no inbound video at all), and the web
raises no desktop notifications.

Assisted-by: Claude:opus-5-5
Run agents were briefed from a scratchpad file that named today's
AppImage and libtacky and the old branch. The template keeps its hard
rules (path: flake, no rm, no passwords, look at every screenshot, one
pushed `vm:` commit per run, markers_missing, cause) with placeholders
for server, peers, specs and builds, takes the features from PLAN.md,
and adds a mandatory last phase: whatever the agent scripted goes into
examples/, clients/ or interopvm.py, its lessons into AGENTS.md, the
touched example is re-run once, and the report says what stayed manual.
AGENTS.md (previous commit) points to it first.

Assisted-by: Claude:opus-5-5
Merge origin/main into vms/consolidate
All checks were successful
/ check (pull_request) Successful in 4m23s
Details
83f0df43bd
 
Brings PLAN.md (#12), which RUN-BRIEF.md now points to, and prosody's
STUN/TURN (#13); AGENTS.md keeps main's prosody calls line beside the
intermittent Dino audio note.

Assisted-by: Claude:opus-5-5
asxpi merged commit 88dcbd7fbb into main 2026-11-09 08:26:52 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
another-im/interop!14
No description provided.