Eight research-grounded AI resident personas evaluated ayenay.org across three rounds — 19 structured sessions in total — producing five findings, a decision matrix, and a build-ready roadmap. This report shows the results and the safeguards behind them.
The bell converts visits. The privacy answer converts votes. Every panelist who might return needs to be fetched — a topic alert, a ping, a heads-up before the decision. And every panelist who might actually cast a vote needs one plain sentence about who can see it. Two different conversions, at two different moments — and the website currently ships neither.
Beyond that headline, four things stood out across two visit waves and a moderated group round:
SYNTHETIC · DIRECTIONAL — the default for every panel finding. All 8 personas, all 19 sessions. Where a count appears (e.g., 6/8), it means that many of the eight personas independently produced the finding.
CROSS-CORROBORATED — 6 findings independently reproduced when the same website capture and equivalent persona specs were run through a second, unrelated AI system. Procedure in §03.
REAL-USE OBSERVATION — 1 observation, from the researcher's own live session on the site. A single disclosed data point that corroborates a panel finding; it does not outrank the panel, it grounds it.
ASSUMPTION-DEPENDENT — the return-wave findings, which assume new agenda items appear before council votes (never yet observed). Flagged inline wherever it applies.
Can: surface likely friction points, trust barriers, and adoption mechanics before real users hit them; rank fixes by how many distinct resident types each one unlocks; pressure-test the site against the town's most skeptical audiences on demand.
Can't: replace real residents. Summary accuracy against full staff reports was not adjudicated (source documents weren't in the capture) — accuracy verdicts here are structural, not substantive. All quantified claims ("7 of 8") count simulated personas, not people. Treat every finding as a strong, evidence-backed hypothesis.
The value of a synthetic panel lives or dies on its machinery. What follows is the architecture, not the blueprint — enough to judge how much weight the findings can bear. The full protocol behind it is proprietary and stays with the researcher.
Every persona was constructed against layered real-world evidence: Morro Bay census data (median age ~54, a third of the town 65+, ~41% renter households, a working waterfront), the local record itself — council minutes, agenda archives, regional reporting on the town's live controversies — and published national research on civic participation and technology adoption, used to keep each persona's behavior inside empirically observed ranges rather than stereotype. Where the data ran out, assumptions were documented, not smuggled in.
Eight seats, each answering one question no other seat can (§04) — and deliberately skewed skeptical, with half the panel low-trust by design, because a panel that loves the website produces nothing usable. Composition was balance-audited across age, gender, housing tenure, income, and attitude before a single session ran.
Synthetic panels have three known failure modes: sycophancy (AI's instinct to flatter), convergence (eight voices quietly collapsing into one), and grounding drift (reacting to things a real first-time visitor could never see). Each had dedicated countermeasures built into the session design, and every round was audited against all three after the fact — output that failed audit was struck. A two-persona pilot gate ran first; the full panel wasn't committed until the pilot passed clean.
Cold first visits ran on each persona's real attention budget — some sessions capped at four distracted phone-minutes, because that's the session real residents give a new website. Personas encountered the site the way residents actually do: through the town's own word-of-mouth channels, not in a vacuum. Second visits came two weeks later in scenario time, novelty gone, measuring what actually pulls anyone back. A moderated group round closed the run — top agreements, standing divisions, and the one change that moves the most seats from "maybe" to "yes" — with minority positions preserved at full strength, never averaged away.
Every session ended with a structured challenge that forces the persona to attack its own construction — a designed-in skepticism mechanism aimed at the panel itself. It earned its keep: several of this report's best findings came out of it, including Jess's "the ping is the product," Marisol's point that real-world abandonment is silent and never shows up in anyone's data, and Doug's observation that the launch-state problems precede the AI question entirely.
The same website capture and equivalent persona specifications were independently run through a second, unrelated AI system. Its outputs were audited before any comparison, and findings it produced without grounding in the capture were excluded. Only findings both systems reached independently carry the CROSS-CORROBORATED tag — six qualified. Divergences between the systems are logged, not hidden: findings unique to one system carry lower confidence until validated.
Assumptions, declared: the return wave assumed the website ingests new agendas before council votes (its stated design; never yet observed — ASSUMPTION-DEPENDENT), no product changes in the interval, and a routine agenda with no hot-button item. If any assumption fails in reality, the findings leaning on it are voided — and that dependency is flagged at every point of use in this report.
Each card names the seat's single question. Expand any card for how that persona was built and what it was designed to test.
Does Ayenay add anything for someone who already reads every packet?
Can a third of the town physically and comfortably use this?
Does the convenience promise land for the actual target user — in 90-second phone sessions?
Does it survive a hostile sixty-second audit — and answer "who's really behind this?"
Does it save time for the economically motivated — pure utility, no ideology?
Does the website read as neutral ground on the town's most polarized issue?
Can the AI summaries survive someone who knows exactly how they fail?
Does "community voting" signal belonging to the 41% who rent — or read as the owners' room?
All personas are composites; no real individual is depicted.
Six panelists — through six different fears — converged on one question. The researcher's own live session then hit the same question unprompted. When that many roads lead to one spot, that spot is the website's fault line.
The four personal-exposure fears are answered by a plain visibility statement. The two aggregate-integrity fears are answered by posted voting rules and honest denominators. One "we take privacy seriously" line satisfies neither. Collapsing this into "users worry about privacy" would destroy exactly the texture that makes it fixable.
During the live capture session — a real, signed-in session conducted by the researcher — a comment was posted on the Police Accreditation item, and the researcher's contemporaneous note on the capture reads: "who can see this?" To be precise: that question was the researcher's own real-time reaction while using the site, not text the website displayed. It is one data point, from one user, who is also the report's author — disclosed as such. Its value is narrow but real: a genuine human, mid-use, asked the exact question the panel converged on, triggered by the same interface asymmetry the panel flagged. It corroborates the finding; it does not independently establish it.
"It says anonymous and it says public in the same sentence. Pick one and say it big."— BARBARA (SIMULATED), GROUP ROUND
At evaluation, every feed item carried the stamp "Council voted Jun 23, 2026." Seven of eight panelists hit this independently in Round 1 — each for a different reason — and in the group round it became the panel's single 8-of-8 agreement, each seat naming it in its own currency.
"You built me a ballot box when I asked for a doorbell."— JESS (SIMULATED)
ASSUMPTION-DEPENDENT Return-wave findings assumed new agenda items now appear pre-vote. The cheapest, highest-value check available: watch the feed the day the next agenda posts, against the city's own alert timing.
In the return wave, five of eight panelists had contact with the site. Zero touched the voting layer. Every return targeted a non-voting surface — and every return was triggered by something outside the website.
The pattern: the roll-call accountability tracker — the page that asks nothing of anyone and names the powerful instead — quietly retained an auditor, a hostile skeptic's respect, and the panel's most exposure-averse member. The voting layer, the website's namesake, retained no one. That's not a failure to hide; it may be the website's actual center of gravity.
"Accountability that runs toward city hall instead of away from it — that, I'd take."— RAY (SIMULATED), THE PANEL'S HARDEST NO
Six of eight found the plain-language summaries genuinely good — for several, the most readable civic writing they'd ever been handed. "Is it AI?" was mostly a non-event. What broke trust was never the prose; it was what surrounds the prose.
The standing minority position (both auditors, preserved at full strength): readable is not verified. "The better it reads, the less anyone checks." No summary has yet been graded against a contested item — that grade is pending, and it will happen publicly whether Ayenay prepares for it or not.
The website's central phrase is undefined, and three panelists interrogated it from mutually exclusive positions. This is not a UX bug. It's Ayenay's constitutional question, and only the operator can answer it.
Owns here, pays substantial TOT here, can't vote in city elections. "Am I the community or the revenue?" If excluded: taxed, regulated, voiceless again. If included: the first STR item becomes a scoreboard — and if owners use it, "we're the astroturf story." Either way, she forwarded the site to her network within the hour, and they now monitor it weekly. The first faction to discover an unclaimed civic platform recruits on it.
"Resident" is never defined; nothing says renters count, nothing says they don't — and she's learned what that silence usually means. She also noticed something bigger: the word-of-mouth chatter about the website reproduced, person for person, the room it was meant to replace. "They built a new room and the same people found it first."
"A third retirees and a fifth second homes, voting on my livelihood — majority of whom?" His demand isn't inclusion; it's that a vote total never be quotable as "Morro Bay supports X" without visible eligibility rules. Otherwise the tool isn't neutral ground — it's "a number waiting to be weaponized in someone's staff report."
In the group round, all three agreed on exactly one thing: the rules must be posted on the door, in writing — who may vote, one-per-person or not, and what "resident" means. The answer matters less than its existence.
A diagnostic panel is built to find problems — so when a deliberately skeptical panel hands out credit, it's earned. These four positives survived contact with the panel's hardest graders.
Barbara — the panel's comprehension stress test — read an entire infrastructure item and understood it: "clearer than the letter they'd send about it." Marisol: "easier than anything the county ever mailed me." For a civic website, solved readability is a real, rare asset. The AI writing layer is doing its job.
It earned the domain auditor's daily habit ("my hand-built spreadsheet, automated"), forty full seconds from the panel's most hostile seat, and the only voluntary return visit from its most exposure-averse one. It asks nothing of the user and names the powerful. Strongest page on the site.
"Voting records are pulled automatically from meeting captions and marked unofficial until the minutes are published" — both auditors independently praised this line as exactly the right epistemics. Frank: "more sourcing discipline than most staff reports show." Direct staff-report links on detail pages passed structural checks. This transparency instinct, extended to the AI summaries themselves, is the trust play.
The panel's single successful conversion: Jess found the City Park town halls (with Zoom links) on the calendar, screenshotted them, and attended one — the exact outcome a civic engagement site exists to produce. The loop never closed (she credits her camera roll, not the site), but the value delivery was real. The mechanism works; it just works invisibly.
| Panelist | Create an account? | Return next week? | Trust the AI summaries? | What flips them |
|---|---|---|---|---|
| Frank | NO | YES — daily, tracker only | WITHHELD | Beat his city email alert to one agenda; survive one contested-item audit |
| Barbara | NO | NO | READS FINE — wary | A trusted neighbor posting "I did it, it took two minutes, it's fine"; a rate item surfaced before the letter |
| Jess | NO | NO | "SURE" — beside the point | Topic notifications. Full stop. "The ping is the product." |
| Ray | NO | NO | NO VERDICT — never saw a harbor item | A name on the About page; harbor-item alerts routed to his wife's phone |
| Pam | NOT YET | ONCE — grapevine-fetched | FINE UNTIL WRONG | Four keywords and a bell — "I'd pay actual money for it" |
| Denise | NO | WEEKLY — adversarial watch | DEFERRED to first STR item | Posted voting rules; an STR summary both sides recognize themselves in |
| Doug | NO | YES — once per test | NO | A methods & corrections page with a named human; real accounts; one clean contested-item diff |
| Marisol | NO | ROLL-CALL PAGE ONLY | "WASN'T THE BARRIER" | Ironclad anonymity in plain words; one rent item, surfaced early, framed for renters |
YES · NO · CONDITIONAL / QUALIFIED — note the pattern: every YES in the return column is attached to a non-voting surface, and account creation is currently impossible regardless (sign-in is demo-only), which suppresses the entire first column and reads to skeptics as an unfinished website soliciting donations.
Each of these is a natural follow-on module — see §14.
Everything above came from a standing capability: eight calibrated resident perspectives that can be pointed at anything Ayenay ships, changes, or writes — on demand, in hours. Two tracks from here.
The findings are strong hypotheses; real Morro Bay residents get the last word. The validation sequence, in order of cost: (1) the one-agenda-cycle check — watch the feed the day the next agenda posts, which settles the pre-vote assumption for free; (2) put real residents on the same tasks the panel ran and score where they confirm, contradict, or surprise; (3) publish the hit/miss ledger — misses documented as loudly as hits, which is what turns this from a clever simulation into a calibrated instrument.
Every UX or copy change re-run against the eight seats before it ships. "Does the new privacy line actually satisfy Pam and Marisol?" is answerable in an afternoon, before real users ever see it.
Before the first STR, parking, or harbor summary goes live, run it through Denise, Ray, and Frank. If any faction can't recognize its own argument, fix it before the town grades it publicly. This defuses the report's #1 risk.
The same engine renders to whatever the moment needs: a one-page brief for a council conversation, a donor-facing summary, an onboarding-copy review, a features one-pager — same evidence base, different documents.
Seats can be added (a second-homeowner, a city-staff perspective, a Spanish-first household), rules adjusted, and new modules bolted on — the not-covered list in §13 is the menu. The panel gets sharper every cycle it runs.
This round answered "what's confusing, what's missing, what should it do." The standing panel answers the harder question every week after launch: "before we ship this — how does Morro Bay read it?"