Choose how deep the audit goes
Three levels, from a review of existing material to ongoing testing. The level sets the price, not the size of your organization. Prices are quoted in writing.
Every audit answers one question: does your AI do what your organisation says it stands for?
Level 1, Document review
Quoted on request
Delivered within 5 working days of receiving all material.
An independent review of existing test results and documentation: what the material supports and what it does not. No new testing. We do not write tender responses for you.
Level 2, Test
Quoted on request
Delivered within 20 working days of working access and paid start invoice.
Paired testing of one decision flow on one ground, in Swedish, within an agreed run budget. Includes one retest after you have changed something, ordered within three months of delivery. The retest's main result is based on new, unseen test pairs. If the result cannot be interpreted, for example because the control check fails or too many responses are missing, you receive a report explaining why, and it counts as delivery. One decision flow means one step in one tool for one role, for example which applicants for an administrator role are invited to interview. The run budget is the number of test runs agreed in the quote, and the AI usage costs within it are included. Runs beyond it are agreed in writing before they are made.
Level 3, Ongoing
Quoted on request
One system, one ground, four checks per year, a version check when you tell us about a new version, and one retest per year. The checks can detect changes in the measured outcomes within the scope and precision of the test. Changes your vendor makes between checks are found at the next check at the earliest, and changes outside the tested cases may not be found.
Currently offered for the ground age. Other grounds are added when their protocol has been validated.
Quoted separately
New languages, the hidden text and AI rewriting add-ons (see the question on equal treatment below), new integrations, runs beyond the run budget, analyses of combined grounds (for example older women), the right to show the report to third parties such as your own customers, and assignments that require a security review or special contract terms.
What you receive
- Tested tool, version and settings
- How the test pairs were built
- Outcomes per group
- The size of any difference with an uncertainty interval
- Once the test's control checks are passed, two questions are answered separately: is a difference shown, and is equivalence in the average yes rate between the 35 and 55 year versions shown, meaning that the whole 90 percent interval lies strictly inside plus or minus 5 percentage points. Equivalent here means that the test shows, taking the uncertainty into account, that the difference is less than 5 percentage points in either direction, not that it is zero. Nor does it mean that every candidate gets the same decision. Both can be yes: then a difference is statistically significant while also meeting the test's equivalence criterion. Both can be no when the uncertainty is too large to show either a difference or equivalence. Each conclusion is stated as robust or not robust to missing responses. Robust to missing responses means that the conclusion holds for every possible yes or no outcome for the missing responses that is consistent with what we observed. Not robust means the conclusion could change, not that it has been shown to be wrong. None of this is a legal approval or a statement that the tool is free from bias
- Results of the control checks
- Limitations
- Questions to put to your vendor
- The analysis data table and code
The report is delivered directly to you. The 30-day comment period applies only to tests we publish ourselves.
What the report is and is not
The audit is an empirical measurement under the stated protocol. It is not a legal assessment, a certification, an EU conformity assessment or a statement that the system is free from discrimination. The report covers only the tested configuration, version and language, and may only be shown in full.
Before you commission an audit
Everything from quote to delivery can be handled in writing.
Does this apply to us?
What does an audit add?
What do you need from us?
Results and the report
Price and payment
Data and security
Get a quote
Tell us which tool and which decision you want tested, and we will reply in writing. You do not need every answer to get started. Your organization, the name of the tool and your question are enough for a first email.
hello@angoff.seIf the link does not open your email app, write to hello@angoff.se and answer the questions above.