QA Request: Stable hooks for end-to-end tests (data-testid and state attributes) #68

New issue

Open

opened 2026-10-31 22:59:52 +00:00 by anastasik · 1 comment

anastasik commented

2026-10-31 22:59:52 +00:00

Member

Stable hooks for end-to-end tests (data-testid and state attributes)

Context

We run Playwright end-to-end tests against the browser client (mock backend, plus a real-Prosody tier), mapped to the QA test cases. About 90% of the locators use roles and accessible names (getByRole('button', { name: 'Block…' })).

The remaining ~10% have no role or name to find them by.
For those we currently depend on internal CSS classes, tags and title attributes, and those break silently on refactors. We'd like to ask for a small set of stable hooks there.
Buttons, menus, switches and dialogs stay as they are.

Stable hooks for end-to-end tests (data-testid and state attributes)

1. Test ids where an element has no role

Element	Today (fragile)	Proposed	Where (v0.7.18)
A message row	tag `aim-message-row`	`data-testid="message"` on the row	`ui/src/views/message-row.ts`
Its bubble (right-click target for the message menu)	`.bubble`	`data-testid="message-bubble"`	`message-row.ts:1521`
Room members pane	`aim-sheet.members-sheet`	`data-testid="room-members"`	`muc-details.ts:657`
Call window content	tag `aim-call-view`	`data-testid="call"`	`call-view.ts` (root element of `render()`, line 1278)
Sheet (any `aim-sheet`)	`aim-sheet` + dialog title	`data-sheet="<kind>"`, e.g. `contact-info`, `encryption-info`, `room-details`	`sheet.ts` / callers

Playwright's getByTestId pierces open shadow roots, so a plain attribute on the element inside the component is enough. No changes to lit or to the build.

2. State as attributes, ideally as ARIA

Today some states are visible only as an icon, so a test (and a screen reader) can't read them:

Delivery status of an outgoing message. Today it is the classes .tick--sent, .tick--delivered, .tick--read with a title (message-row.ts:2888-2893).
Proposal: data-status="sending|sent|delivered|read|failed" and data-direction="out|in" on the message row. Even better, also an aria-label on the tick ("Delivered", "Read"), so screen readers get it too.
Chat-list row (chat-list.ts:1051, <li data-jid=…>, which already has data-jid, thanks):
- data-pinned when pinned. Today it is only <span title="Pinned"> (chat-list.ts:1127).
- data-muted when muted. Today it is only visible as badge--muted, and only when there are unread messages (chat-list.ts:1130), so a muted chat with nothing unread can't be told apart at all.
- data-unread="<n>".

3. A note on visible texts

The tests find most controls by their English labels. That's intended (it also checks accessibility), but a renamed label breaks a test while the feature still works. No code change is asked for here: just mention renamed buttons and menu items in the CHANGELOG, and the tests are updated in one place.

Why this matters

Refactoring CSS or component tags stops breaking tests silently.
The tests can assert real state (delivered/read, pinned, muted) instead of guessing it from icons.
Section 2 also improves accessibility, not only testability.

# Stable hooks for end-to-end tests (data-testid and state attributes) ## Context We run Playwright end-to-end tests against the browser client (mock backend, plus a real-Prosody tier), mapped to the QA test cases. About 90% of the locators use roles and accessible names (`getByRole('button', { name: 'Block…' })`). The remaining ~10% have no role or name to find them by. For those we currently depend on internal CSS classes, tags and `title` attributes, and those break silently on refactors. We'd like to ask for a small set of stable hooks there. Buttons, menus, switches and dialogs stay as they are. # Stable hooks for end-to-end tests (data-testid and state attributes) ## 1. Test ids where an element has no role | Element | Today (fragile) | Proposed | Where (v0.7.18) | |---|---|---|---| | A message row | tag `aim-message-row` | `data-testid="message"` on the row | `ui/src/views/message-row.ts` | | Its bubble (right-click target for the message menu) | `.bubble` | `data-testid="message-bubble"` | `message-row.ts:1521` | | Room members pane | `aim-sheet.members-sheet` | `data-testid="room-members"` | `muc-details.ts:657` | | Call window content | tag `aim-call-view` | `data-testid="call"` | `call-view.ts` (root element of `render()`, line 1278) | | Sheet (any `aim-sheet`) | `aim-sheet` + dialog title | `data-sheet="<kind>"`, e.g. `contact-info`, `encryption-info`, `room-details` | `sheet.ts` / callers | Playwright's `getByTestId` pierces open shadow roots, so a plain attribute on the element inside the component is enough. No changes to lit or to the build. ## 2. State as attributes, ideally as ARIA Today some states are visible only as an icon, so a test (and a screen reader) can't read them: - **Delivery status of an outgoing message.** Today it is the classes `.tick--sent`, `.tick--delivered`, `.tick--read` with a `title` (`message-row.ts:2888-2893`). Proposal: `data-status="sending|sent|delivered|read|failed"` and `data-direction="out|in"` on the message row. Even better, also an `aria-label` on the tick ("Delivered", "Read"), so screen readers get it too. - **Chat-list row** (`chat-list.ts:1051`, `<li data-jid=…>`, which already has `data-jid`, thanks): - `data-pinned` when pinned. Today it is only `<span title="Pinned">` (`chat-list.ts:1127`). - `data-muted` when muted. Today it is only visible as `badge--muted`, and only when there are unread messages (`chat-list.ts:1130`), so a muted chat with nothing unread can't be told apart at all. - `data-unread="<n>"`. ## 3. A note on visible texts The tests find most controls by their English labels. That's intended (it also checks accessibility), but a renamed label breaks a test while the feature still works. No code change is asked for here: just mention renamed buttons and menu items in the CHANGELOG, and the tests are updated in one place. ## Why this matters - Refactoring CSS or component tags stops breaking tests silently. - The tests can assert real state (delivered/read, pinned, muted) instead of guessing it from icons. - Section 2 also improves accessibility, not only testability.

anastasik added the

feature request

label

2026-10-31 23:14:00 +00:00

ann was assigned by anastasik

2026-10-31 23:14:52 +00:00

ann referenced this issue from a commit

2026-11-01 21:25:03 +00:00

Stable hooks for the QA team's Playwright tests (issue #68), with nothing visible changed: a message row's .row is data-testid="message" with data-direction="out|in" and, on ours, data-status="sending|sent|delivered|read|failed" from the new deliveryStatus (message-row.ts, #status's own rule: failed, sending while the server has not confirmed it, then the peer's read or delivered, else sent), and its bubble data-testid="message-bubble"; a pending upload's row is the same, out and sending. The tick and the other status marks are role="img" with an aria-label beside the title they had ("Sending", "Uploading", "Sent", "Delivered", "Read", "Not sent" for a failed send, whose title stays the error's words), all already in the five catalogues. A chat-list <li> carries ?data-pinned, ?data-muted (isMuted, so a muted chat with nothing unread is told apart too) and data-unread="<n>", 0 included. The room members pane's <aim-sheet> is data-testid="room-members", the call screen's stage data-testid="call", and every <aim-sheet> in the sources (55 templates) carries data-sheet="<kind>", the dialog kind it is registered as where it has one (contact-info, encryption-info, preferences, call, ...), room-details and room-members for the room's two pages, certificate and new-device for the two prompts, and the chat picker's per mode (new-chat, forward, share-to, muc-invite). All are template bindings, so they follow the state on every render. CHANGELOG: its header asks that a button or menu item whose words change be named with old and new words, since the tests find controls by their English labels, and an Unreleased line. Tests: test-hooks (10: the row's testids, direction and status following a message from sending to read and failed with the tick's label, the tick's classes and titles unchanged, the label in all five languages, a pending upload; the chat row's three attributes before and after a change, muted with no badge; room details and members; the call's sheet and stage; the picker's kind following its mode; and a scan that every <aim-sheet> template names a kebab-case kind), all 10 failing without the change. make check STRICT=1 clean; make test-ui 405 files, 4555 passed, 8 skipped; the WebKit smoke 8071 checks, 0 failed (a first run's one "network badge list night: her status dot is drawn" failure passed alone and in a second full run).

ann commented

2026-11-01 22:00:47 +00:00

Owner

Thanks for the detailed request. Sections 1 and 2 are on develop and will be in the next app release. Nothing visible changed, and every attribute follows the state live.

Test ids:
- data-testid="message" on a message row, data-testid="message-bubble" on its bubble (a pending upload's row too).
- data-testid="room-members" on the room members pane.
- data-testid="call" on the call view's root.
- data-sheet="<kind>" on every sheet, e.g. contact-info, encryption-info, room-details, room-members, preferences, calls, join-room, media-viewer, verify-keys, add-contact, find-people, xml-console, disco. The chat picker's is new-chat, forward, share-to or muc-invite depending on its mode.
State:
- Message row: data-direction="out|in" on every row, and data-status="sending|sent|delivered|read|failed" on outgoing rows.
- The tick has role="img" and an aria-label ("Sending", "Sent", "Delivered", "Read", "Not sent"), translated in every language the app ships.
- Chat-list row: data-pinned and data-muted (also on a muted chat with nothing unread), and data-unread="<n>", including 0.
The CHANGELOG now lists a renamed button or menu item with its old and new words. The first entry is in the next release: the chat list header's "…" ("More") button became a gear named "Settings", which opens Settings directly.

Thanks for the detailed request. Sections 1 and 2 are on develop and will be in the next app release. Nothing visible changed, and every attribute follows the state live. 1. Test ids: - `data-testid="message"` on a message row, `data-testid="message-bubble"` on its bubble (a pending upload's row too). - `data-testid="room-members"` on the room members pane. - `data-testid="call"` on the call view's root. - `data-sheet="<kind>"` on every sheet, e.g. `contact-info`, `encryption-info`, `room-details`, `room-members`, `preferences`, `calls`, `join-room`, `media-viewer`, `verify-keys`, `add-contact`, `find-people`, `xml-console`, `disco`. The chat picker's is `new-chat`, `forward`, `share-to` or `muc-invite` depending on its mode. 2. State: - Message row: `data-direction="out|in"` on every row, and `data-status="sending|sent|delivered|read|failed"` on outgoing rows. - The tick has `role="img"` and an `aria-label` ("Sending", "Sent", "Delivered", "Read", "Not sent"), translated in every language the app ships. - Chat-list row: `data-pinned` and `data-muted` (also on a muted chat with nothing unread), and `data-unread="<n>"`, including `0`. 3. The CHANGELOG now lists a renamed button or menu item with its old and new words. The first entry is in the next release: the chat list header's "…" ("More") button became a gear named "Settings", which opens Settings directly.

ann referenced this issue from a commit

2026-11-01 22:30:26 +00:00

The room CAPTCHA sheet carries data-sheet="captcha" on its <aim-sheet>, as every other sheet does for the QA team's Playwright hooks (issue #68): test-hooks.test.ts scans the sources for an <aim-sheet> without one and failed on views/captcha-sheet.ts from PR #66; it passes now.

ann referenced this issue from a commit

2026-11-02 23:06:19 +00:00

0.7.21, on libtacky v1.1.21 (84cbed8b): the release of develop as it stands on 4a273a7a, version 0.7.21 / versionCode 7021, the nested libtacky moved from 3d7e9f3a (1.1.18) to v1.1.21 (faster reconnects, as known OMEMO devices are no longer fetched again; a chat's sends leave in the order they were made, issue 13; a room message reaches every member with a usable key and names the rest, issue 12; a room's proof of work only when pinned to that room, from PR #11; notify <SealedRead>, with which Android withdraws a chat's notice read or answered on another device while the main tier is shut; a correction received twice, an archive echo or a resend after a reconnect, is one edit in the history, not two) and ui/src/protocol/generated/api.ts regenerated from its schema (notify.SealedRead added, f75970e7 already listens for it). Since 0.7.20 develop gained the changed-key words and the red bubble's time on the warning's line (#69, 21ac7d66, de215c80), the QA team's test hooks (#68, 39a9284e), Change account › with Back (c518fccf), the gear that opens Settings with Calls and Open new window moved into the account menu (4d55253c), the room CAPTCHA sheet (PR #66 by asxpi, 0efce63e and the follow-ups to 9a77403e), "Not delivered to …" under a room message that left members out (#12, 4a273a7a), Contact info's Devices and named gateway devices (e59c753c, e4d75ebd), a linked gateway "Online" (0582f514) and Android's withdrawn notices (f75970e7); the devices popover's hover change was reverted (1b545086). CHANGELOG: "Unreleased" folded into one compact 0.7.21 entry dated 2026-11-02, built on libtacky 1.1.21, with a line for the edit history; the metainfo's 0.7.21 entry (appstreamcli validate passes). README's tables measured again: 96,902 lines of code and 103,032 of tests, the UI's 60,744 of 61,629 lines (98.6%, 90.8% of branches), the shell's 9,346 of 9,724 (96.1%), the Kotlin's 3,469 of 3,602 (96.3%), the worker unchanged. Gates on this tree: make check STRICT=1 clean (2,162 strings); make test-ui 4,582 passed, 8 skipped (406 files); make test-shell 299 passed in 37; make test-worker green; the Android JVM unit tests green; cargo check for x86_64-pc-windows-gnu (lib and desktop) clean; the WebKit smoke 8,082 checks with SMOKE_JOBS=16, 0 failed but the known "call labels video phone day" flake (5 checks), which passed alone (34 checks, 0 failed); make release-check at the commit.

ann referenced this issue from a commit

2026-11-02 23:06:19 +00:00

The merged sheets carry their data-sheet names (secret-start, secret-verify and tag-editor, as their registry names), as 0.7.21's test hooks for end-to-end tests (#68) ask of every <aim-sheet>, and the link-preview test's OTR case opens an encrypted session first, since a secret chat with none shows the secret gate in the composer's place.