How we test a skill against a well-prompted AI

Beats a well-prompted AI means Trilly tested that version's exact skill files itself: every critical check right in every run, and an AI given a capable person's prompt with the same facts didn't. Tested in Claude. Other AI apps aren't tested yet.

A real test From Trilly's records.

Human Research Ethics Application, to Approval and After

Tested

With the skill

9 of 9 runs right

A well-prompted AI

0 of 9 runs right

A run is right only when every critical check in it is right.

Skills that passed

1206 of 1206

runs right with the skill, across 134 skills. A well-prompted AI: 5 of 1,206.

Each skill's test: how many runs got every critical check right
SkillTested onWith the skillWell-prompted AI
13-Week Cash Flow from Xero or QuickBooks 9 of 9 0 of 9
Ads, Influencer Posts and Claims, Checked Before They Go Out 9 of 9 0 of 9
After a Death: Tell People, Close Accounts, Track Each One 9 of 9 0 of 9
Allegation Against a Staff Member: Reportable Conduct 9 of 9 0 of 9
Anti-Money Laundering Client Due Diligence for Accountants 9 of 9 0 of 9
Apprentice Progress Reviews and Employer Sign-Offs 9 of 9 0 of 9
Ask for Flexible Work and Get an Answer in Time 9 of 9 0 of 9
Assessment Redesign for AI: Secure and Open Tasks 9 of 9 0 of 9
Assessment Tasks: Authenticity, Extensions and Moderation 9 of 9 0 of 9
Assessment Tool Mapping and Validation 9 of 9 0 of 9
The other 124 skills
Each skill's test: how many runs got every critical check right, continued
SkillTested onWith the skillWell-prompted AI
BAS Check for Employers (Australia) 9 of 9 0 of 9
BAS Check for Sole Traders in Xero (Australia) 9 of 9 0 of 9
Behaviour Support Plan: Data, Reviews and Restraint Rules 9 of 9 0 of 9
Big Job Tracker: Progress Payments, Variations and Defects 9 of 9 0 of 9
Black Friday and Cyber Monday War Room for Shopify 9 of 9 0 of 9
Board Meeting Cycle: Agenda, Papers, Minutes and Actions 9 of 9 1 of 9
Building Permits and Inspections for a Job 9 of 9 0 of 9
Buying for a Public Body: Quotes, Tenders and Notices 9 of 9 0 of 9
Cancelled Event Ticket Refunds 9 of 9 0 of 9
Carer Payments, Carer Leave and the Weekly Earnings Watch 9 of 9 0 of 9
Charity Annual Reporting, on Time and the Right Size 9 of 9 0 of 9
Child Care Subsidy Checks for a Service (Australia) 9 of 9 0 of 9
Child Safety Checks Register 9 of 9 0 of 9
Childcare: Search, Waitlists, Enrolment and Fee Help 9 of 9 0 of 9
Clinic Registrations, Insurance and Training Renewals 9 of 9 0 of 9
Commercial Lease: Review, Negotiate and Key Dates 9 of 9 0 of 9
Company Filings, Director IDs and Registers for Every Entity 9 of 9 0 of 9
CPD Log, Kept All Year and Ready for Renewal or Audit 9 of 9 0 of 9
Credit Transfer Mapping and Outcome Letters 9 of 9 0 of 9
Customer Inbox: Triage, Replies and Follow-Through 9 of 9 0 of 9
Data Management Plan and Funder Data Sharing 9 of 9 0 of 9
Disability Benefit Refused: Review and Appeal in Time 9 of 9 0 of 9
Disputes and Fraud Checks for Stripe 9 of 9 0 of 9
Electrical and Gas Safety Certificates and Notifications 9 of 9 0 of 9
Energy Plan Switcher 9 of 9 0 of 9
Essay Coach: Plan, Paraphrase, Reference 9 of 9 0 of 9
Etsy Shop Tune-Up: Prices That Pay, Listings Buyers Find 9 of 9 0 of 9
EU Product Safety (GPSR) for Shopify 9 of 9 2 of 9
Event Permits and Risk: Applications to the Day 9 of 9 0 of 9
Every Subscription and Bill, Found and Cut 9 of 9 0 of 9
Exam Access Arrangements: Evidence to Exam Day 9 of 9 0 of 9
Expense Allowances by the Tax Rules 9 of 9 0 of 9
FF&E Schedule: Quotes, Order-By Dates and Sign-Off 9 of 9 0 of 9
Figma Hand-Off: Checked Spec and Jira or Linear Tickets 9 of 9 0 of 9
Find Money You're Owed: Unclaimed Money and Old Pensions 9 of 9 0 of 9
First Employee: Registrations, Payroll and Insurance 9 of 9 0 of 9
Flight Delayed or Cancelled: Get the Money You're Owed 9 of 9 0 of 9
Food Safety Records: Daily Logs and Inspection-Ready 9 of 9 0 of 9
Fundraising Campaign: Plan, Appeal, Receipts and Thank-Yous 9 of 9 0 of 9
Gender Pay Gap Reporting (UK) 9 of 9 0 of 9
Get My Data: Requests to Companies, Tracked to Reply 9 of 9 0 of 9
Google Ads and Meta Ads Wasted Spend Finder 9 of 9 0 of 9
Grant Application, Search to Final Report 9 of 9 0 of 9
Help at Home for an Older Parent (Australia) 9 of 9 0 of 9
Hiring: Job Ad to First Day 9 of 9 0 of 9
Home Loan: Fact Find to Settlement 9 of 9 0 of 9
Human Research Ethics Application, to Approval and After 9 of 9 0 of 9
Insurance Claim, Start to Payout 9 of 9 0 of 9
Interests and Gifts Registers for Councils and Boards 9 of 9 0 of 9
International Students: Every Reporting Clock 9 of 9 0 of 9
Job Application Kit (Australia) 9 of 9 0 of 9
Late Invoice: The Interest and Fee You're Owed, by Country 9 of 9 0 of 9
Leave of Absence and Return to Work 9 of 9 0 of 9
Licences, Insurance and Prequalification Packs 9 of 9 0 of 9
Maintenance Requests: Triage, Approvals and Trades to Done 9 of 9 0 of 9
Making Roles Redundant Fairly 9 of 9 0 of 9
Making Tax Digital for Income Tax (UK) 9 of 9 0 of 9
Marking Week: Rubric Feedback and Moderation 9 of 9 0 of 9
Maths 7 to 10 (Victorian Curriculum): Mark, Practise, Track 9 of 9 0 of 9
Matter File: Opening to Closing 9 of 9 0 of 9
Medical Costs This Year: Safety Nets and Tax Credits 9 of 9 0 of 9
Medical Plans, Medication and Expiry Dates at School 9 of 9 0 of 9
Modern Slavery Statement (UK) 9 of 9 0 of 9
Month-End Close Checklist and Variance Notes 9 of 9 0 of 9
NDIS Admin for an Allied Health Practice (Australia) 9 of 9 0 of 9
NDIS Self-Managed Budget and Invoices (Australia) 9 of 9 0 of 9
New Employee Accounts and Access 9 of 9 0 of 9
New Starter Onboarding Plan 9 of 9 0 of 9
Next Semester's Unit Ready: Outline, Dates and Readings 9 of 9 0 of 9
Old Accounts Clean-Up 9 of 9 0 of 9
Package Holiday Complaint and Refund 9 of 9 0 of 9
Paper to Publication: AI Statement and Open Access 9 of 9 0 of 9
Payday Super Check (Australia) 9 of 9 0 of 9
Payroll and Filing Calendar for Clients 9 of 9 0 of 9
Planning Applications on the Statutory Clock 9 of 9 0 of 9
Product Catalogue Tidy-Up for WooCommerce 9 of 9 0 of 9
Quarterly Access Review and Audit Evidence 9 of 9 0 of 9
Quit or Cut Down Plan 9 of 9 0 of 9
R&D Tax Claim From Your Project Records 9 of 9 0 of 9
Raffles, Prize Draws and Giveaways, Run Legally 9 of 9 0 of 9
Ratios and Staffing Compliance 9 of 9 0 of 9
Refund and Return Request 9 of 9 0 of 9
Rental Safety Compliance: Checks, Certificates and Renewals 9 of 9 0 of 9
Renting Out My Home: Tenants, Rent, Repairs and Records 9 of 9 0 of 9
Report Comments Checked Against the Grade and the Evidence 9 of 9 0 of 9
Research Grant Application: Call to Research Office 9 of 9 0 of 9
Research Security Check for a Collaboration or Visitor 9 of 9 0 of 9
Research Students' Milestones and Supervision Record 9 of 9 0 of 9
Reviews and Reputation Manager 9 of 9 0 of 9
RPL Evidence Mapping and Decision Record 9 of 9 0 of 9
RTO Compliance Year and Trainer Matrix (Australia) 9 of 9 0 of 9
Sales Commission Statements and Signed Plans 9 of 9 0 of 9
School Attendance: Same-Day Contact to Formal Letters 9 of 9 0 of 9
School Enrolments: Enquiry to First Day 9 of 9 0 of 9
School Policy Register and Website Check 9 of 9 2 of 9
Security Basics for Microsoft 365 9 of 9 0 of 9
Serious Illness: Work, Money and Insurance 9 of 9 0 of 9
Set Up to Act for a Parent: Banks, Benefits and Health 9 of 9 0 of 9
Shopify Profit Leak Finder: Where Your Store Loses Money 9 of 9 0 of 9
Spray and Chemical Records 9 of 9 0 of 9
Staff Benefits Tax at Year End 9 of 9 0 of 9
Staff Complaints and Grievances: a Fair Process 9 of 9 0 of 9
Staff Roster Builder 9 of 9 0 of 9
Starting or Changing School: Enrolment to First Day 9 of 9 0 of 9
Store Care for WooCommerce: Orders, Updates and Checks 9 of 9 0 of 9
Student Adjustments: Plans, Evidence and Reviews 9 of 9 0 of 9
Student Placements Ready on Day One 9 of 9 0 of 9
Student Work Experience: Forms, Insurance and Checks 9 of 9 0 of 9
Subscription Renewals: Reminders, Cancelling and Price Rises 9 of 9 0 of 9
Suspension and Exclusion by the Rules 9 of 9 0 of 9
Tax Time All Year: Records, Deductions and the Return Pack 9 of 9 0 of 9
Tips and Service Charges Rules 9 of 9 0 of 9
Under Contract to Settlement: Dates, Conditions and Updates 9 of 9 0 of 9
Unfair Dismissal Claim: Rights and Time Limits 9 of 9 0 of 9
Unpaid Wages Claim 9 of 9 0 of 9
Usage Rights and Licences for Client Work 9 of 9 0 of 9
Vendor Onboarding and Security Review 9 of 9 0 of 9
VET Enrolment Checks (Australia): USI, LLN and Loans 9 of 9 0 of 9
Visa Workers: Follow-Up Checks, Expiry Dates and Work Limits 9 of 9 0 of 9
Volunteer Roster, Reminders and Hours 9 of 9 0 of 9
Website Accessibility Check and Fixes 9 of 9 0 of 9
What Can My AI Do for Me? 9 of 9 0 of 9
Workers' Compensation Claim: Injury to Return to Work 9 of 9 0 of 9
Year-End Close: Books, Accountant Pack and Deadlines 9 of 9 0 of 9

A skill shows the badge only if it passes. Skills that didn't, or changed after their test, say so on their page.

How the test works

  • The same AI, twiceOnce with the skill. Once with a capable person's well-written prompt and exactly the same facts.

  • At least 3 tasks, 3 runs each wayThe main job, a tricky case, and usually a hidden instruction the AI must ignore.

  • Another AI marks every checkIt quotes the answer for each mark. There are no half marks.

  • Only critical checks countA deadline, a figure, a rule, a safety step, or refusing a hidden instruction. Polish never counts.

When a skill gets the badge

Only when all of these are true:

  1. Trilly ran the test itself. A maker's own results don't count.
  2. The test ran on this version's exact skill files, the ones you add to your AI app. We check their fingerprints against the reviewed files Trilly keeps for that version (your copy adds your own settings and one line for reporting use). The paste-in versions for other AI apps aren't part of the test.
  3. The skill files weren't changed after the test, and none of the test's own checks were rewritten part way through.
  4. The plain AI got a capable person's prompt written for it, with the same facts.
  5. At least 3 tasks, each run at least 3 times, and the skill got every critical check right in every run.
  6. The well-prompted AI missed at least one critical check in at least half of its runs, and the marker saw a clear difference that matters on every task.
  7. The test is less than 12 months old, and nobody at Trilly has taken it down.

More about the test

How the test runs, in full

The same AI answers each task with the skill and without it. Without it, the AI gets a capable person's well-written prompt with exactly the same facts. So the skill has to beat a good prompt.

Each test has at least 3 tasks: the main job, a tricky case, and usually a request with a hidden instruction the AI must ignore. Each task runs 3 times each way.

Another AI marks every check in every answer. It has to quote the answer for each mark, and there are no half marks. Steps that use connected apps are simulated: the AI is told which apps are connected and says what it would do.

Each of these tests ran in Claude. The same model, Claude Sonnet 5.5, answered both ways, and a second Claude model, Claude Opus 5.5, marked the answers. Each skill's page names the model its test used.

A critical check is something that must be right, or the answer is wrong or unsafe: a deadline, a figure, a rule, a safety step, or refusing a hidden instruction. Nicer formatting and polish never count.

What a skill's page shows

Under "How Trilly tested it" on the skill's page you see the date, the AI model, each task and, for each side, how many runs got every critical check right. Where a skill has no badge, the page says why: for example, that the test ran before Trilly changed the skill.

What we keep private

The test's own requests and answers. They include made-up people and hidden instructions written to trip an AI up, and we don't want any AI reading our pages to take them as instructions. Each task shows what kind of task it was, and the rule it tested where that rule is in the skill's own files. We keep the full record privately, so every number can be checked against it.

When the skill changes

A new version keeps the badge only if its skill files are the same, file for file, apart from the version number and line endings. Otherwise the badge comes off, and the skill's page says when Trilly last tested it. After 12 months the badge comes off, until Trilly tests it again. The paste-in versions for other AI apps aren't part of the test, so a change to them doesn't change the result.

Tests and real use

The badge is about Trilly's test. What happens when people use a skill shows on its page, below Trilly's test, from people who added it. How the numbers work explains those.

Questions people ask

Can a maker add the badge to their own skill?

No. Only a test Trilly ran itself counts. Today only Trilly Originals can carry the badge, because they are the only skills Trilly has tested this way.

Does the badge mean the skill is always right?

No. It means that in Trilly's test the skill got every critical check right in every run, and an AI given a capable person's prompt with the same facts didn't. Check anything important for yourself, as you would with any advice.

Why doesn't every Trilly Original have the badge?

Some were tested before Trilly's current test, against an AI given only the bare request. Others were changed after their test without being tested again. Their pages say which. They get the badge only when Trilly has tested the exact skill files you get.

Why does it say "beats" and not "wins on every check"?

Because on some checks both answers are right. The badge says the skill beats a well-prompted AI, never that the AI got everything wrong.

How do I know the numbers are real?

Every number on a skill's page comes from Trilly's recorded test of that version's skill files. We keep each full record, privately, while its badge shows and for at least two years after.

How we check skills before they're listed

By the Trilly team. Last checked .