Transition Monitor — Business Analysis Case Study

The problem

The method lived in one analyst's head.

Terrawatt's analysts produce a Country Transition Scorecard on request — how fast is this country's electricity system actually decarbonising, and who is stalling. The output mattered. The process producing it did not survive contact with a second analyst.

What actually happened

An analyst downloaded open energy data into Excel and hand-cleaned it — units, missing years, country name variants
They applied a scoring method that existed only in one senior analyst's head
They emailed the workbook to a reviewer, who emailed comments back
Roughly 15 working days per country, with no audit trail
Two analysts scoring the same country produced different answers
When the source published an update, every scorecard silently went stale

What that costs

Key-person risk: the methodology is undocumented and unversioned
No defensible answer to "why is this country scored 62?"
Review by email means no record of what changed, or why
Published numbers quietly diverge from the data behind them
The bottleneck is a person, so the service cannot scale

Process design

Before and after, in BPMN 2.0

Both diagrams are modelled in Enterprise Architect using the same conventions, so they are directly comparable. Flip between them: the analyst lane empties out, the system lane fills up, and the review gate survives the redesign on purpose.

BPMN collaboration diagram of the Transition Monitor scorecard process — AS-IS — the manual process. Every annotation is a real failure mode: “method lives in one analyst's head — key-person risk”, “email ping-pong, no audit trail”, and a System lane whose only job is to let scorecards go stale.

Click a diagram to open it full size.

What the redesign changes

A new data release triggers the cycle, instead of triggering nothing
Validation runs before scoring, and quarantines what fails rather than silently dropping it
One scoring engine, so two analysts get the same answer
Every scorecard is versioned with an audit trail as it is created
Review happens in the system — no email ping-pong, no lost comments

What the redesign deliberately keeps

The human review gate. Automation makes the draft; a person still approves it
The analyst's ability to comment and adjust before submitting
A change-requested path back to the analyst, rather than a hard reject
Publication as an explicit act, not a side effect of a successful run

The data model

Why every published number can be traced back

A star schema in UML: one fact, four dimensions. The parts that matter for governance are the flags and the version dimension — they are what let the process above make promises it can keep.

FactCountryScore is grained one row per country × indicator × date × version. DimScorecardVersion carries status and version number, which is what makes the approval workflow and the audit trail possible. DimIndicator holds each component's weight, direction and scoring method as data — so changing a weight is a business decision with a record, not a code change.

Click to open it full size.

Data validation

One metric, three answers

A star schema only protects you from a bad structure. It says nothing about whether the numbers going into it are trustworthy. Before scoring a single country, I checked whether the data even agreed with itself.

Renewable electricity share for Germany, Lithuania, Poland, Slovenia and Sweden — 2023 — pulled from three independent sources: Our World in Data, IRENA, and Eurostat. Same metric name, same year, same countries.

Bar chart: renewable electricity share by country and source, 2023. Sweden, Germany, Slovenia and Poland show close agreement across Eurostat, IRENA and OWID; Lithuania shows a 48-point spread (36%, 74%, 84%). — Same country, same year, three different answers — the largest gap is 48 percentage points.

Click to open it full size.

What it found

Four of the five countries agreed closely — within 2–3 percentage points, well inside normal reporting variance. Lithuania did not. The same country, the same year, reported as 36%, 74%, and 84% depending which source you asked — a 48-point spread, larger than the gap between the highest- and lowest-scoring country in the entire dataset.

A dashboard that silently picks one of those three numbers is not being neutral. It is being confidently wrong.

What I did about it

Disclosed it, rather than resolving it or hiding it. The instinct with an inconvenient outlier is to quietly exclude it. I decided the divergence itself was the more useful finding — a live example of why a single-source dashboard needs a documented system boundary before anyone trusts its numbers.
Defined the system boundary explicitly (D-012): Transition Monitor measures renewable electricity generation share — not consumption, not the broader energy balance. That definitional choice is what makes cross-source and cross-country comparison honest instead of coincidental.
Logged the reversal, not just the decision. An earlier version of this decision (D-011) proposed excluding Lithuania while its data was under-explained. Once the full three-source comparison was in, D-013 formally superseded that — the wrong instinct is on the record alongside the right one, because that's what an honest decision log looks like.

D-013 — Include Lithuania as a documented divergence case, not an exclusion

Excluding an inconvenient country is the easy version of data governance. Disclosing exactly where and why sources disagree is the harder, more useful version — and it's the one a reviewer can actually act on. Status: Agreed. Supersedes D-011. Impacts the Data Quality Assessment, the published country set, and the scoring methodology's system boundary.

From requirement to screen

The wireframe carries its requirement IDs

A low-fidelity screen is only useful if you can say which requirement each element satisfies. Every callout here points back to a numbered functional requirement — which is also how the traceability runs the other way, from requirement to design to test.

Low-fidelity wireframe of the Transition Monitor scorecard view, annotated with functional requirement references — Scorecard view (Analyst). **FR-42** filtering and search · **FR-08** the “requires refresh” flag, answering the AS-IS pain point where a scorecard silently goes stale · **FR-28/43** governance flags carried straight from the data model · **FR-04/05** version history visible on every published scorecard.

Click to open it full size.

Requirements & governance

Every requirement traces to a story and a test

The diagrams are the visible part. The chain underneath them is what makes the work usable by a delivery team — and what lets anyone ask, a year later, why the system does what it does.

The traceability chain

20 business requirements (BR-01 to BR-20), each mapped to a TO-BE process activity and an epic
43 functional requirements (FR-01 to FR-43), each with Given / When / Then acceptance criteria
5 epics and 14 stories in Jira, carrying real issue keys — not placeholder IDs
A reconciled BR → FR → Epic → Story → Test table, updated when the built backlog came in at 14 stories rather than the 20 originally planned

The governance layer

Stakeholder register, RACI matrix and power/interest grid — with the automation gains visible as activities where R moves from Analyst to System
A glossary covering domain, modelling and delivery terms, so BRD, FRD and backlog use one vocabulary
A decision log: thirteen entries, statuses of Open, Agreed and In Progress, each with rationale and the artifacts it affects
Documentation in Confluence, backlog in Jira, models in Enterprise Architect

Three decisions from the log

A decision log is only useful if it records the reasoning, not just the outcome. Two of the thirteen. are still open, which is what a live project actually looks like.

D-004 — Component weights are configuration, not code

Weights change when the methodology changes. Storing them as a governed configuration value rather than a constant means a weight change has an owner, a date and a reason — and “it’s in the code somewhere” is never the answer to why a country scored 62. Status: Agreed. Impacts BR-12, FR-27 and the data model.

D-008 — Keep the fictional client, but disclose it

The scenario needed a named client for the methodology to cohere. Naming one without saying so would misrepresent the work. So the decision to disclose is itself logged, with the disclosure line specified in the project overview. Status: Agreed.

D-009 — Defer the Power BI report to Phase 2

The data-preparation groundwork is complete, but the report is a BI deliverable rather than a business-analysis one. Deferring it deliberately — and recording that it was deferred — is more honest than leaving it looking unfinished. Status: Agreed.

Still open

D-001 — should nuclear count as “clean”? Parked pending research, for methodology v1.1
D-002 — which regional standard for roll-ups, World Bank or the source’s own?

Methodology

The decisions worth arguing about

The scoring index is a 0–100 composite of five components — clean electricity level, five-year momentum, carbon intensity, fossil dependence and energy productivity. The interesting part is not the formula. It is the handful of decisions that stop the formula lying to you.

Missing is not zero

A country needs at least three of the five components to be scored at all. Fewer, and it returns Insufficient Data — a status, not a low score. Treating a gap as a zero quietly punishes countries for their statistics office rather than their power sector.

Shares are recomputed, never averaged

Renewable share is SUM(renewable TWh) / SUM(total TWh). Averaging a percentage across countries or across years weights a country generating 4 TWh the same as one generating 400. This is the single most common way an energy dashboard becomes confidently wrong.

Outliers are capped before normalising

Energy productivity is capped at the 95th percentile before the scale is built, so one unusual economy cannot compress everyone else into the bottom third of the range.

Micro-states are scored but not ranked

Below 1 TWh of annual generation, a single new installation moves the score more than a decade of policy. They still get a score; they are excluded from the league table.

Weights are a business decision, not a constant in the code

They live in a versioned decision log and in the indicator dimension. When Research changes a weight, the change has an owner, a date and a reason — and every scorecard built under the old weights stays reproducible.

Scorecard lifecycle

Modelled as a UML state machine, because the rules are easier to argue about as states than as prose.

Draft→ In Review→ Approved→ Published→ Superseded

↩ From In Review a reviewer can send it to Changes Requested, which returns to In Review once the analyst responds — a loop, not a dead end.

A scorecard may only be published from Approved. When a new data vintage lands, the previously published scorecard becomes Superseded — never deleted. That one rule is what makes an old published number defensible a year later.

Transition Monitor — turning a 15-day manual scorecard into a governed process

The method lived in one analyst's head.

What actually happened

What that costs

Before and after, in BPMN 2.0

What the redesign changes

What the redesign deliberately keeps

Why every published number can be traced back

One metric, three answers

What it found

What I did about it

D-013 — Include Lithuania as a documented divergence case, not an exclusion

The wireframe carries its requirement IDs

Every requirement traces to a story and a test

The traceability chain

The governance layer

Three decisions from the log

D-004 — Component weights are configuration, not code

D-008 — Keep the fictional client, but disclose it

D-009 — Defer the Power BI report to Phase 2

Still open

The decisions worth arguing about

Missing is not zero

Shares are recomputed, never averaged

Outliers are capped before normalising

Micro-states are scored but not ranked

Weights are a business decision, not a constant in the code

Scorecard lifecycle

Success criteria I'd hold myself to

Delivery

Governance

Recognise this process?