Skip to content
American Institute of Business & TechnologyAmerican Institute ofBusiness & TechnologyAmerican Institute of Business & Technology home

Methodology

How each track is built

Published in full, because an assessment whose method is not inspectable is not an assessment. It is an opinion with a number attached.

Five stages, three of them before anyone can take it

A track is not a variant of one master questionnaire with the nouns swapped. Each is researched, written and validated against the standards governing its own profession, which is why they reach publication at different times.

01

Framework research

Pre-publishing

Each track is anchored to the regulatory and professional body frameworks governing its own target population. No cross-sector borrowing.

  • Review of the professional body AI guidance specific to the track, such as ABA Formal Opinion 512 for legal, FSB and IOSCO for financial services, NACD and OECD for board leadership, or C2PA for creative and media.
  • Competency domains derived from those standards rather than from a generic template.
  • Target population and exclusion criteria defined before any item is written.
  • Literature review of existing measurement instruments in the field.
  • The theoretical construct documented and agreed before drafting begins.
02

Expert item development

Pre-publishing

Subject matter experts draft 40 items per track, calibrated to observable professional behavior rather than to abstract knowledge.

  • 25 behaviorally anchored rating-scale items distributed across the competency domains.
  • 15 scenario-based judgment items concentrated in the highest-stakes areas.
  • Three proficiency levels per item: Acquire as the safe minimum, Deepen as competent applied practice, Create at expert and governance level.
  • Anchors describe what practitioners at each level actually do, not what they know in the abstract.
  • Reviewed by the track subject matter lead before it reaches the expert panel.
03

Content Validity Index review

Pre-publishing

An independent panel scores every item on relevance and clarity. Nothing enters a published instrument without passing this gate.

  • A minimum of six subject matter expert judges per track, independent of the item authors, following Lynn (1986) and Polit and Beck (2006).
  • Every item rated separately on two criteria: relevance to the construct, and clarity of wording.
  • Item-level (I-CVI) and scale-level (S-CVI/Ave) indices computed on the four-point ordinal scale established by Lynn (1986).
  • Acceptance threshold for the full instrument: S-CVI/Ave at or above 0.90.
  • Items below threshold on either criterion are revised, not removed, and resubmitted for a second independent round.
04

Track publishing

Launch

The validated instrument is integrated into the platform and the track opens to organizations.

  • Scoring rubric and level thresholds locked. No post-publishing item change without re-validation.
  • The track becomes available to organizations holding an applicable access code.
  • Response data begins accumulating from licensed professional respondents.
  • Volume monitored in preparation for the psychometric stage.
05

Psychometric validation

Ongoing

As responses accumulate, the instrument is tested formally for factor structure and internal consistency.

  • Exploratory Factor Analysis once 150 or more responses per track are available.
  • Confirmatory Factor Analysis at 300 or more responses, to confirm the domain factor structure.
  • Internal consistency measured against a Cronbach's alpha target of 0.80 or above per domain subscale.
  • Item performance and score drift monitored between formal cycles.
  • Periodic refinement as the professional and regulatory landscape moves.

What content validity does and does not establish

It establishes that the items represent the construct the instrument claims to measure, as judged by experts in the field. It does not establish reliability, construct validity, or responsiveness. Those require accumulated response data, which is what stage five is for.

What we will not claim

A track that has passed CVI review is content-validated. It is not, and is not described as, psychometrically validated until the analysis in stage five has actually been run on it. That term covers the full set of properties, and claiming the whole from the part is the kind of thing a serious buyer checks.

On self-report

There is a fair criticism of self-report instruments: they can capture confidence rather than ability, and confidence is a poor proxy. Two things address it here.

Used this way the instrument is a baseline and placement tool, which is what it is good at, rather than a measurement of demonstrated capability, which it is not and does not claim to be.

Framework alignment

Results report against the UNESCO AI Competency Frameworks, the UNESCO Recommendation on the Ethics of Artificial Intelligence, the AI Fluency 4D Framework, and the U.S. Department of Labor AI Literacy Framework, plus the professional body guidance specific to each track. The frameworks are the measurement and reporting layer. They are not the syllabus, and courses are not organized around them.

Not affiliated with UNESCO

This content is not created by, affiliated with, or endorsed by UNESCO. Alignment to a published framework is not certification by its author. The UNESCO frameworks are licensed CC BY-SA 3.0 IGO and are free to reference with attribution.

Sources

The expert panel

Panels are recruited openly. Researchers, academics and credentialed professionals in a track area can apply to review; a cycle runs roughly 40 to 60 minutes and scores items on relevance and clarity.