Estay Dynamics
AI Expectations Gap Map

Methodology and data updates

How each technology is scored, how the board projects different futures, and how the data is kept current. Numbers on this page are read from the same data files the board uses, so they always match what you see there.

1. Why not a hype cycle

The hype cycle places each technology on one curve against an unlabelled time axis. It has three weaknesses for investment decisions.

  • One axis carries two meanings. A position on the curve mixes how much attention a technology gets with how mature it is. The two often move independently: foundation models are commoditising while attention stays high; knowledge graphs deliver value with little attention.
  • No mechanism. The curve is a shape, not a model. It cannot say what would move a technology faster or slower, so it cannot be used to test scenarios.
  • Weak empirical record. Reviews of the model (Dedehayir and Steinert, 2016) and retrospectives of past cycles (Mullany, 2016) found that most technologies do not follow the full curve. Many appear once and vanish.

This board separates the two quantities, makes the dynamics explicit, and exposes every assumption in an editable file.

2. What the board measures

Each technology has three scores from 0 to 100.

Expectations vertical axis
How much the market believes the technology will deliver, and how soon. It shows up as attention, investment announcements, valuations and executive agendas. High expectations are not a problem in themselves; they are a problem when evidence does not support them.
Evidence horizontal axis
How much value the technology has demonstrably delivered in production, outside the vendors selling it. Production deployments, measured financial impact, independent trials and failure rates count. Demonstrations, pilots and vendor claims count for little.
Impact bubble size
How much business value the technology would create if it worked as promised. Impact does not move the bubble. It decides how much a wrong call costs, and it weights the correction-risk ranking.
Gap
gap = expectations − evidence. A positive gap means the market is ahead of the evidence and a correction is possible. A negative gap means the technology delivers more than it is credited for.

3. Scoring rubric

Analysts score against these anchors. Scores between anchors are interpolated. Each score is recorded in data/assessments.csv with the review date and a one-line rationale.

Expectations
ScoreAnchorTypical observable signs
0–20NicheDiscussed mainly in research papers or specialist forums. No dedicated vendor category.
20–40Specialist interestEarly-stage startups, conference tracks, occasional trade-press coverage.
40–60Mainstream business topicLarge vendors market it, analyst firms cover it, appears in business press regularly.
60–80Board-level priorityLarge funding rounds and capex commitments, CEO statements, valuations that assume success.
80–100Treated as inevitableCapital and public policy committed ahead of results. Sceptics are a small minority in public debate.
Evidence
ScoreAnchorTypical observable signs
0–20LaboratoryResearch results and demonstrations. No paying production use.
20–40PilotsPilots and a few production deployments. No independent measurement of financial impact. High pilot cancellation.
40–60Early productionEarly adopters in production with quantified case studies. Results vary widely between organisations.
60–80Repeatable valueWidespread production use, repeatable returns, mature vendors and established practices.
80–100InfrastructureStandard practice or commodity. Value is assumed and competition is on price.
Impact (if it works)
ScoreAnchor
0–20Marginal efficiency gain in one activity.
20–40Material improvement within one function.
40–60Material improvement across several functions, or a meaningful cost line.
60–80Changes the operating model of an industry.
80–100General-purpose technology with economy-wide effects.

Weighting evidence sources

When sources conflict, rank them in this order and state in the notes which one drove the score:

  1. Independent measurement: randomised or controlled studies, audited financial results, regulator data.
  2. Observed behaviour: production deployment counts, renewal and cancellation rates, spend data.
  3. Practitioner reports with numbers: case studies that state baseline, intervention and result.
  4. Surveys of intent or satisfaction.
  5. Vendor claims and press coverage. These inform expectations, not evidence.

4. Classes and actions

Each technology falls into one of four classes based on its gap and evidence. The thresholds live in data/settings.json.

The diagonal band on the map is the region where expectations and evidence roughly agree. The class decides the recommended action, which follows real-options logic: commit capital in proportion to evidence, and buy cheap options where uncertainty is high.

5. Collected signals

A monthly job collects four public signals per technology. They need no API keys and are reproducible by anyone. Queries per technology are configured in data/technologies.json.

SignalSourceWhat it measuresUsed for
Page viewsWikimedia REST API, English Wikipedia, human traffic onlyBroad public attentionExpectations
Story countsHacker News via Algolia search, query in story titlePractitioner and technical attentionExpectations
MomentumChange in page views and story counts against the prior three monthsWhether attention is rising or fallingExpectations
Paper countsarXiv API, query in any fieldResearch activityShown only
Downloadspypistats.org, last month, configured packagesDeveloper usageEvidence proxy

Time windows

Each refresh uses the last three complete calendar months as the current window and the three months before as the prior window. A run on 3 November reports August to October. Three months smooths one-off news spikes.

From raw counts to indices

Raw counts differ by orders of magnitude between technologies and sources, so each component is log-scaled and normalised across the set:

nc(i) = ( ln(1 + xc(i)) − minj ln(1 + xc(j)) ) / ( maxj − minj )

Momentum is the average log ratio of current to prior window across Wikipedia and Hacker News, then min-max normalised the same way.

attention index = 100 × ( 0.4·nwiki + 0.4·nhn + 0.2·nmomentum )
usage index     = 100 × npypi

When a component is missing for a technology (no Wikipedia article, no package), its weight is redistributed over the components that exist. A component is only computed when at least three technologies have it.

The indices are relative. 0 means the least attention in this set and 100 the most. If attention to all AI rises together, the indices stay flat. The raw counts and momentum in each detail card show absolute change.

6. Blending judgement and signals

The board offers three ways to set each technology's starting position, selected under "Starting positions".

Analyst
Scores from assessments.csv only.
Blend default once signals exist
A weighted mix of analyst scores and indices.
Signals only
Indices wherever they exist, analyst scores elsewhere. Useful to see what the market's attention alone says.
H₀ = (1 − wH)·analyst expectations + wH·attention index
E₀ = (1 − wE)·analyst evidence     + wE·usage index

The evidence weight is deliberately lower. Attention is a reasonable proxy for expectations; downloads are a weak proxy for value. Evidence remains mainly a matter of analyst judgement against the rubric.

7. The projection model

The projection is a two-stock system dynamics model, solved with Euler steps ( per year). It is deliberately small: every parameter can be explained in a sentence and changed in a file.

Evidence grows logistically

dE/dt = reff · E · (1 − E/100)
reff = max(0, r · (1 + Σd sd · Dd))

Evidence accumulates fastest at mid-maturity and slows near saturation. r is the technology's base growth rate per year. Dd is the setting of driver d (from −1 to +1) and sd is the technology's sensitivity to that driver. A sensitivity of +0.7 to capability progress means a strong capability scenario (+1) raises the growth rate by 70%.

Expectations adjust toward evidence plus a premium

dH/dt = k · ( clamp(E + P, 0, 100) − H )
P = a·(impact − 50) + b·Dcapital + c·Dcapability

Expectations are a stock that closes the gap to a target at rate k, the hype correction speed. The target is current evidence plus a narrative premium P. The premium is higher for high-impact technologies (people tolerate a larger gap for bigger prizes), when capital is abundant, and when capability progress is strong. A capex correction scenario turns the premium negative and pulls expectations below evidence.

Parameters in use
SymbolMeaningValueSet in

What the model does not do

  • Evidence never falls. A technology that fails outright shows up as expectations collapsing toward a stagnant evidence level, not as evidence declining.
  • Technologies do not interact. Progress in foundation models does not automatically lift agents. Shared drivers (capability, cost) stand in for those links.
  • Drivers are constant over the horizon. A scenario is a sustained condition, not a shock in a given year.

8. Worked example

Projection of one technology under the base case.

YearExpectationsEvidenceGapClass

9. Drivers and scenarios

Five drivers describe the environment. Each runs from −1 (much weaker than today) to +1 (much stronger). Zero means current trends continue.

DriverMeaning

Presets combine driver settings into named futures. They are stress tests, not forecasts: the useful question is which technologies change class across scenarios, because those are the bets most exposed to the future you cannot control.

10. Limitations

  • Judgement drives the result. Starting scores, sensitivities, thresholds and the premium formula are analyst choices. The board makes them explicit and testable; it does not make them objective.
  • Attention is not adoption. The attention index measures looking and talking. It is a fair proxy for expectations and a poor one for value.
  • Queries are imperfect. Broad terms ("AGI", "LLM") pick up unrelated stories; narrow ones ("AI factory") miss relevant ones. Review flags in each refresh are partly a check on query quality.
  • Usage proxies are biased. PyPI downloads overweight open-source Python ecosystems and automated installs. They are not available for most technologies.
  • English-language sources. Wikipedia (English), Hacker News and arXiv over-represent US and English-speaking technical communities.
  • Gartner phases are for comparison only. They appear in detail cards and are not used in any calculation.

11. How the data is updated

Two kinds of data are updated on different cycles, with different levels of automation. Signals are collected by machine and reviewed by a person. Scores are set by a person and checked by machine. Nothing reaches the live board without a merged pull request.

Monthly · automaticSignal refresh

3rd of each month, 05:17 UTC. Collects signals, opens a pull request.

Quarterly · promptedAnalyst review

1st day of each quarter. An issue lists stale scores and a checklist.

As needed · manualStructural changes

Add technologies, tune queries, adjust sensitivities, scenarios or thresholds.

Monthly signal refresh

  1. Collect. The AI Expectations Gap Map signal refresh workflow runs scripts/update_signals.py. It queries each source for every technology, with retries and the pacing each API asks for. It takes 5 to 10 minutes.
  2. Safeguard. If more than half of the requests fail, the job stops and changes nothing. Individual failures, such as a missing Wikipedia article, are recorded and the rest proceeds.
  3. Compute. The script calculates the indices, writes data/signals.json, and adds a snapshot for the month to data/history.json. A rerun for the same month replaces that month's snapshot.
  4. Validate. scripts/validate_data.py checks every data file.
  5. Propose. The workflow opens a pull request titled "AI Expectations Gap Map data: signal refresh for YYYY-MM". Its description lists the largest moves in the attention index, every technology where signals and the analyst score differ by more than points, and any failed requests.
  6. Review. For each flagged technology, decide whether the signal reflects a real shift. If it does, edit data/assessments.csv on the pull request branch, update the score and set reviewed to today. If the signal is noise, leave the score and consider tuning the query.
  7. Publish. Merge the pull request. The tests run again, and GitHub Pages publishes the board within a few minutes.

To refresh outside the schedule, open Actions → AI Expectations Gap Map signal refresh → Run workflow. You can enter a specific month (YYYY-MM) to backfill.

Quarterly analyst review

  1. On the first day of each quarter, the AI Expectations Gap Map quarterly review workflow opens an issue listing every technology whose score has not been reviewed in 120 days.
  2. Read the last three refresh pull requests and their review flags.
  3. Re-score each technology against the rubric, using the evidence hierarchy. Record the main reason in notes.
  4. Set reviewed to today on every row you confirmed or changed, including rows that did not change. The date records that someone looked.
  5. Open a pull request with the changes. The AI Expectations Gap Map tests workflow must pass before merging.
  6. Close the issue with a link to the merged pull request.

Editing scores on GitHub

Open data/assessments.csv on GitHub, select the pencil icon, edit, and choose "Create a new branch and start a pull request". Each row has this format:

id,expectations,evidence,impact,reviewed,notes
ai-agents,82,35,90,2026-10-24,High cancellation risk; coding and service agents deliver.

Notes that contain commas must be wrapped in double quotes. The validator rejects scores outside 0–100, malformed dates, duplicate rows and ids that do not exist in technologies.json.

Structural changes

Add a technology
Add an entry to data/technologies.json (id, name, base rate, five sensitivities, watch signal, signal queries) and a matching row in data/assessments.csv. Validation fails if either is missing.
Tune a query
Edit the signals block of the technology. Prefer specific phrases over single words. Check the next refresh for the effect.
Change sensitivities or thresholds
Edit technologies.json or settings.json. Record the reason in the pull request description, because these change every projection.
Add a scenario
Add a preset to data/scenarios.json with a unique id, a name, a value for each driver and a one-sentence note.

Roles

RoleResponsibility
Data ownerMerges refresh pull requests within a week; owns query quality; runs the workflow by hand if a scheduled run fails.
Analyst reviewersScore technologies quarterly against the rubric; resolve review flags; propose new technologies.
Model ownerApproves changes to sensitivities, thresholds, premium constants and scenarios.

One person can hold all three roles. Separating at least the analyst and model-owner roles reduces the risk of tuning the model until it confirms a view.

When something goes wrong

No pull request this month
Check the Actions tab. A failed run usually means a source API was down; rerun it by hand. If the workflow could not open a pull request, check that Actions may create pull requests (Settings → Actions → General).
Many failed requests for one source
The source may have changed its API. The other sources still work; the affected component is dropped and its weight redistributed.
A technology jumps implausibly
Usually a query matching an unrelated event. Do not change the score; tune the query.
The board shows an error
A data file failed to load or parse. Validation should prevent this; check the latest AI Expectations Gap Map tests run.

12. Keeping score

Each monthly refresh also freezes the board in data/ledger.json: the analyst scores and signals as they stood, the starting position the board showed, and the model's forecast for every preset scenario 3, 6, 12 and 24 months ahead. A record is never rewritten, so the forecasts cannot be adjusted after the fact.

When a forecast comes due, the track record compares it with where the board placed the technology in that month, and with the simplest rival: a forecast that nothing moves. The model shows skill only where its error is smaller than standing still. It also counts how often the forecast put a technology in the right quadrant, and how often its predicted move in expectations matched the move in the collected attention signals, which do not depend on the analyst.

A later position includes analyst scores that may not have been re-reviewed since the forecast was made; those carry over unchanged and favour standing still. The track record shows how many comparisons had a fresh analyst score.

13. Finding new technologies

The board measures only the technologies it lists, so each monthly refresh also looks for what it is missing. It reads the titles of AI stories on Hacker News and of a sample of AI papers on arXiv, and keeps two- and three-word phrases that are rising, that appear in at least two of the last three months, and that the board does not already track. Names with version numbers, companies, products and headline phrasing are left out, and so is a phrase whose Wikipedia article describes an organisation, a person or a product.

Candidates appear on the board under "Rising, not on the board yet", each with its Wikipedia description where one exists, or example titles where none does. Most are noise. A candidate joins the board only when an analyst adds it with scores, a description and a supply chain example; its track record starts then.

14. Data files

FileContentsUpdated by
data/assessments.csvAnalyst scores, review date, rationalePeople, via pull request
data/technologies.jsonNames, Gartner phase, base rate, sensitivities, watch signal, signal queriesPeople, via pull request
data/scenarios.jsonDrivers and preset scenariosPeople, via pull request
data/settings.jsonBlend weights, thresholds, model constants, defaultsPeople, via pull request
data/signals.jsonLatest indices and raw counts per technologyMonthly workflow
data/history.jsonMonthly snapshots of indices, every month keptMonthly workflow
data/candidates.jsonRising terms not on the board, with descriptions, and every month each has been a candidateMonthly workflow
data/ledger.jsonThe board frozen each month: scores, signals, starting positions and every scenario's forecasts, never rewrittenMonthly workflow

15. Update log

Loading.

16. The technologies

What each technology on the board is, and an example of how it applies to supply chain management. The examples show what the technology does, not evidence that it delivers: that is what the evidence score is for.

Loading.

References

  • Dedehayir, O. and Steinert, M. (2016). The hype cycle model: A review and future directions. Technological Forecasting and Social Change, 108, 28–41.
  • Mullany, M. (2016). 8 lessons from 20 years of hype cycles.
  • Sterman, J. D. (2000). Business Dynamics: Systems Thinking and Modeling for a Complex World. McGraw-Hill. Stock adjustment and logistic diffusion structures.