Why documenting data is a discovery and governance exercise
This is the third article in a five-part series on implementing HACT’s UK Housing Data Standards.
The first two articles (Part 1 & Part 2) considered why organisations need to understand their own data and what a useful housing data dictionary should contain.
This article looks at what happens when the dictionary starts to be populated.
A dictionary may begin as an attempt to record what the organisation already knows. In practice, the work often exposes conflicting definitions, hidden reporting logic, inconsistent code lists, uncertain ownership, and system workarounds that were previously understood only by individual teams.
Those discoveries do not mean the exercise has failed. They are some of its most valuable outputs.
A dictionary does not begin with all the answers
A data dictionary is sometimes approached as though the organisation already understands its information and merely needs someone to write it down in a consistent format.
That assumption rarely survives the first few discussions.
The work may begin with an apparently simple field. An analyst asks what it means, who maintains it, and which system provides the authoritative value.
The first person gives a clear answer. A second team describes the field differently. A developer then explains that a key report does not use the field at all because it derives another value through SQL. An operational user adds that the documented process is sometimes bypassed because the system does not support the outcome required in practice.
The organisation has not created a new problem by asking the question.
The different interpretations, workarounds, and hidden rules already existed. The dictionary has made them visible.
A data dictionary brings together the different ways a field is understood and used across the organisation. The disagreement already exists; documenting the field makes it visible and creates a basis for resolving it.
This is why useful dictionary development draws on several forms of evidence:
- System configuration and technical documentation
- Process maps, procedures, and policies
- Reports, interfaces, and transformation logic
- Data profiling and quality analysis
- Operational users, subject specialists, analysts, and developers
Each source may describe part of the picture. The dictionary provides somewhere to connect those perspectives, identify where they agree, and record where decisions are still required.
A worked example: what does Property Type represent?
Consider again an internal field called Property Type, containing values such as House, Flat, Maisonette, Hostel, Garage, Conversion, Shared Ownership, and Temporary Accommodation.
The field name sounds straightforward. It may be mandatory, populated for almost every record, and restricted to values from an approved code list.
A superficial assessment might therefore conclude that the data is complete and ready to map to the relevant part of the standards. A dictionary-led review would go further.
The definition is not sufficient
The existing description may simply state: The type of property.
That does not explain what is being classified.
Is the field intended to describe dwelling form, building use, accommodation type, construction history, tenure, or operational management?
The values suggest that several concepts have been combined.
House, Flat, and Maisonette may describe dwelling form. Hostel may describe use or accommodation arrangement. Garage is a non-residential asset type. Conversion may describe construction history. Shared Ownership is a tenure, while Temporary Accommodation is more likely to describe an operational use or management route.
The field may be complete without being conceptually coherent.
Different teams may use different parts of it
The next question is not only what the values appear to mean, but what the organisation does with them.
The field may be used to filter repairs, identify compliance populations, group stock for reporting, route work to teams, support rent processes, or populate regulatory returns.
Different users may rely on different parts of the code list. A repairs team may use the distinction between houses and flats. A finance report may identify shared ownership. A temporary accommodation service may use the field to locate its managed stock.
That helps explain how several concepts came to coexist. The code list may have expanded over time to meet immediate operational or reporting needs without anyone reconsidering the original definition.
Values may have several origins
The dictionary should then establish how the values are created and maintained.
Some may be selected when the property record is first created. Others may have been migrated from an earlier system. Certain classifications may be populated through an interface, while others are updated manually when a property’s use changes.
Users may also select the nearest available option because the system does not contain a field for the concept they need to record.
An incorrect value may therefore be the predictable consequence of a weak data structure, rather than careless user behaviour.
Related systems may hold different answers
The organisation may already hold separate information about tenure, building use, construction, dwelling type, and operational responsibility.
A property described as a Flat in the housing system might sit within a Hostel in an asset system and be recorded as Temporary Accommodation in a local operational dataset.
Those values may all be valid because they describe different characteristics. The problem arises when they are treated as competing answers to the same question.
Once the internal field has been understood, the organisation may conclude that it cannot be mapped as one complete item. Some values may align with dwelling form, while others relate to building use, tenure, construction, or operational purpose.
The likely outcome is not a single field-to-field mapping. It is a decision to separate several business concepts that have historically been stored together.
The standards have not created the inconsistency. They have provided a clearer model against which it can be identified.
The disagreement already existed
When different teams provide different definitions, it can be tempting to describe the dictionary process as creating debate or slowing progress.
In reality, the process is exposing differences that already influence reporting, services, integration, and decision-making.
A housing team may interpret Occupied as the presence of an active tenancy. Finance may associate it with an open rent account. An operational team may use it to mean that a person is physically resident. A reporting team may classify any property not recorded as void as occupied.
Those meanings overlap, but they are not identical.
If the differences remain undocumented, they may surface later as conflicting reports, failed reconciliations, integration problems, or disputes over which figure is correct.
Writing down the definition forces the organisation to confront the differences earlier and in a controlled setting.
The aim is not necessarily to impose one definition on every use. An organisation may legitimately require several related concepts, provided they are named clearly, defined separately, and used for appropriate purposes.
The dictionary helps distinguish between:
- Several valid concepts that need clearer names
- One concept implemented differently across systems
- A derived classification being mistaken for source data
- A genuine disagreement requiring a governance decision
- An area where the organisation does not yet know enough to decide
Governance becomes specific
Data governance is often described through strategies, committees, policies, and role descriptions.
Those structures matter, but governance becomes tangible when the organisation must make a decision about an individual data item.
Consider a field called Ownership Type.
To document it properly, the organisation must decide what ownership means in that context. It must distinguish legal title from management responsibility, repair obligations, financial treatment, leasing arrangements, and service-charge responsibility.
It must also establish which evidence supports the value, where it is recorded, who can change it, and what happens when legal, operational, and financial records disagree.
If nobody can answer those questions, the issue is not simply that the dictionary contains a blank description.
It is a governance gap.
System ownership is not data ownership
One common source of confusion is the difference between ownership of a system and ownership of the data held within it.
A system owner may be responsible for application availability, supplier management, configuration, access, upgrades, and technical support.
Those responsibilities do not automatically make that person accountable for the meaning of every business field in the system.
A data owner should be accountable for the business meaning, intended use, quality expectations, key rules, and resolution of material conflicts. A data steward may then support day-to-day interpretation, monitoring, documentation, and issue investigation.
The exact arrangement will vary between organisations. In smaller organisations, one team may perform several roles. The important point is that the responsibilities are explicit and proportionate to the significance of the data.
A technical administrator should not become the default owner of a business definition simply because they have permission to edit the code list.
Definitions and code lists need control
A definition written by an analyst is not automatically an approved organisational definition.
It may be well researched and useful, but significant terms need a proportionate route through review and approval.
A practical lifecycle might include:
- Draft
- Under Review
- Approved
- Published
- Superseded
- Retired
The approving route may vary according to the importance of the concept. A local reporting field may require agreement from a product owner and data steward. A critical property, tenant, compliance, or financial concept may require wider cross-functional approval.
Code lists need similar control.
A code list may contain overlapping options, unclear descriptions, local additions, retired values that remain selectable, or catch-all categories used whenever the correct value is unavailable.
Creating a new code may appear to be a minor system change, but it can affect reports, interfaces, validation, historical analysis, and standards mappings.
The organisation therefore needs to know who can propose, approve, create, amend, and retire values, and what happens to existing records when a change is made.
A code list is not governed merely because the system prevents free-text entry.
Hidden business rules become visible
Some of the organisation’s most important data rules may not exist in policy or process documentation.
They may be embedded within SQL queries, stored procedures, interfaces, reports, spreadsheets, system configuration, supplier code, or individual working practices.
For example, a finance report may classify a property as leasehold using a combination of ownership code, account status, agreement records, and administrative relationships.
Important classifications may be created through hidden SQL, reporting, or transformation logic rather than stored directly as source facts.
The resulting classification may be useful, stable, and well established for financial reporting. It may still be a derived value rather than a direct statement of legal ownership.
The dictionary should distinguish:
- The original source fields
- The transformation or calculation
- The resulting classification
- The purpose for which the classification is valid
- The person responsible for approving the rule
Without that distinction, a derived reporting value can gradually become treated as an authoritative source fact.
The same problem arises where a report compensates for weaknesses in source data. Sophisticated logic may produce a more useful output, but it may also conceal problems in the underlying process.
Documenting the rule allows the organisation to decide whether it should remain a reporting transformation, become a governed business rule, or be replaced through improved source data.
A blank owner is a meaningful finding
One of the most revealing cells in a dictionary may be the owner field.
A missing definition tells the organisation that the meaning has not been documented.
A missing owner tells it that nobody has been made accountable for deciding what the meaning should be, keeping it current, or resolving disagreements.
That is often the more significant discovery.
The dictionary should not hide that uncertainty by assigning ownership to the nearest system or technical team. It should record the gap and create a route through which accountability can be agreed.
A missing definition is a documentation gap. A definition with no accountable owner is a governance gap.
The dictionary turns ambiguity into visible governance work. It shows where definitions require approval, where code lists need control, where derived rules need to be documented, and where nobody is accountable for resolving competing interpretations.
However, defining and governing a data item raises another question: how should the organisation determine whether its values are actually good enough?
Part 4 will examine why technical validity is different from accuracy, and why the definition of the data determines the quality rule.
About HACT
HACT is the charity of the social housing sector, supporting innovation, collaboration, and insight across housing. Since 2018, it has worked with OSCRE and organisations across the sector to develop the UK Housing Data Standards, which provide the common framework at the centre of this series. More recently, HACT has established UK HIVE to support sector-wide collaboration, research, development, and implementation.
I would like to acknowledge HACT, OSCRE, and the wider sector contributors whose work has made the standards possible. This series is an independent discussion of the practical questions that can arise when applying them to existing housing systems, processes, and data.