This guidance covers the reasoning behind and the compilation of our house price statistics.
The 2010 National Statistician’s Review of Official House Price Statistics recommended the creation of a “single definitive house price index produced by the official statistics producer community”, replacing the various indices covering the UK at that time. Addressing this recommendation, the producers of official house price statistics worked together to develop the new UK HPI, which is based on a more comprehensive set of data sources covering both cash and mortgage sales.
Registers of Scotland (RoS) has been producing quarterly and monthly house price figures for Scotland for many years and, following user consultation at the time, have continued to publish these statistics in parallel with the monthly UK HPI.
This guide, therefore, provides details of the compilation of both the Scotland element of the UK HPI and the RoS quarterly and monthly statistics. The base data used for the statistics is comprehensive, capturing all changes of ownership, including cash sales, and subject to quality assurance.
View quarterly, monthly and annual house price statistics.
View the monthly UK House Price Index.
See how these statistics differ in our statistics comparison guidance.
UK House Price Index
To deliver the Scottish element of the UK HPI, RoS extracts details of all market value residential sales on a monthly basis. The extraction is based on the date of entry provided in the application for registration and is published approximately six weeks in arrears, i.e. January data is first published in mid-March. Additional markers are added to each line of data to identify mortgage or cash transactions and house type.
The data will be incorporated by the Office for National Statistics into their HPI model. The HPI applies a Hedonic regression model, and includes the following price-determining characteristics:
- Local authority district in GB and housing market area in Northern Ireland
- ACORN area classification variable
- Property type (i.e. detached, semi-detached, terraced, flat, or unassigned if the property type cannot be identified)
- Floor area (metres squared)
- Number of rooms
- New or old property
Information about buyer status (first-time or former owner-occupier) and financing status (cash or mortgage) is also published each month, but have not been included as price-determining characteristics.
The UK HPI is mix-adjusted to allow for the fact that different houses are sold in different periods by annually updating the fixed basket of properties. The resulting mix-adjustment weights will be calculated annually and will be derived using the latest complete year’s worth of transactions.
The average prices will be calculated using a geometric mean, which is deemed to be the most appropriate method for the HPI because it is less distorted by high values and is in line with international best practice.
Due to the volatility of sales figures for Scotland’s local authorities on a month by month basis, particularly in the smaller local authority areas where there may be very few sales, Scotland’s local authority figures are based on 3-monthly moving average. This creates smoother figures that are less influenced by high or low value sales within each 3-monthly period.
RoS house price statistics
Similar to the HPI publication, RoS house price statistics (monthly, quarterly, calendar year and property market report) extracts details of all market value residential sales. However, the data is extracted based on date of registration – the date that the application was received for registration – rather than date of entry. The registration date is generally within a few weeks of the date of sale. The extraction is also limited to applications where the selling price of the property lies between £20,000 and £1,000,000.
The monthly and quarterly statistics are published on the first working day in the second month after the month/quarter that the data relates to. The calendar year statistics are updated on a quarterly basis and available in the quarterly statistics spreadsheet. The property market report is based on financial years is usually updated a few months after the end of the latest financial year.
Registrations with a value of over £1m are excluded to ensure that a single large value sale does not distort the average and also to ensure that sales of a block of properties are excluded. Sales of a block of properties that have been identified from the existence of multiple addresses in a title are also excluded.
Similarly, registrations with a value of less than £20k are excluded from the calculation to ensure that, as far as practical, a low value sale, or the sale of part of the title to a residential property, does not distort the average. The transfer of part of a title may happen when one of the joint owners of a shared property buys out the share of the other owner.
Additional markers are added to each line of data to identify house type. Where the house type cannot be identified, additional methodology is applied to place a proportion of these unassigned properties into a particular house type. Further details about the house type methodology are provided below.
Average prices are calculated using arithmetic mean and median, and published to local authority level within Scotland. House price and house sales data is also made available on statistics.gov.scot where data is available down to data zone level.
Administrative procedures and background data
The majority of the information provided in our data extractions for both the HPI and the RoS house price statistics are derived from data gathered as part of the land registration process. The exception to this is the house type methodology, which is covered in more detail below.
Registration
The house price data extractions come from information supplied to RoS with new applications to register a house sale. Registration occurs when a transfer of title takes place, regardless of the amount of money involved. As part of the registration process an applicant's agent (usually a solicitor) records the ‘consideration’ for which the title has been reassigned. It is possible for the consideration to be nil or below the true market value of the title. In such a case the applicant's agent (or solicitor) notes the basis of the consideration and also provides a value. Below market price transactions such as these are excluded from the statistics.
The registration process will also provide property address details, date of entry to the property, registration date, title number information, land class, deed type, buyer and seller information and details of any applications for registration of a mortgage over the property. RoS can therefore use the information gathered to identify residential sales, cash sales and new builds and the geographical location of the property.
Land class
RoS records six land classes: residential, commercial, land, agriculture, forestry and other. The land class information is provided by the submitting agent on the RoS application form when an application for registration is submitted.
Only residential sales are included in the data extractions for the HPI and RoS house price statistics (although the property market report includes a section on non-residential transactions).
As part of our data validation RoS reclassifies properties where the land class is incorrect, for example commercial premises that have been classified as residential.
Cash and mortgage transactions
Every deed submitted to RoS for registration is classified by deed type. It is possible, therefore, to identify if an application for registration of a mortgage security is received at the same time as, or in the weeks following, an application for registration of a residential sale.
A cash sale is defined as a residential sale for which no mortgage application has been received either at the time the sale is registered or within the period of up to 100 days following the registration of the sale. For example, if an application for registration of a residential sale with no mortgage security deed is received on 20 January and no application for registration of a mortgage security is received in either January, February or March, then the sale is classed as a cash sale.
As the HPI is based on the date of entry rather than the registration date, a deed received for registration in January could have a date of entry in December or earlier. Our revision process means that a property received for registration on 20 January, but with a date of entry in December, is added to the revised data for December and published in March. The cash sale status of this property is then updated in the April publication but is not updated again.
New builds
The majority of new build properties are sales of plots from a builder’s title. RoS can therefore identify sales from major builders’ titles and flag them up as new builds. These sales are subject to a quality assurance process. The figures are likely to include the majority of new build transactions undertaken, but will exclude sales related to very small developments or single new build properties by private or small-scale builders.
Analysis by region
Properties are allocated to local authority area through use of a grid reference applied to an automated Geographical Information System. As a result of incomplete, or wrong, grid references, a small number of properties cannot be allocated to a local authority area. The HPI will exclude these unallocated properties from the analysis, while the analysis within the quarterly statistical release identifies these separately as "Unallocated".
Data quality
Data validation and records excluded from analysis
Prior to producing the data extractions for the HPI and RoS house price statistics, a validation check is carried out on the database of registrations and the following actions are taken:
- Registrations in which the Consideration is less than the Value are identified so that they can be excluded from the extractions. This ensures that the analysis excludes registrations which are not true, market value sales.
- The land class provided is compared manually with the address data with the aim of ensuring correct classification of residential properties.
- Sales of multiple properties in one application are identified so that they can be excluded from the extractions.
- Duplicate lines of data are identified and removed.
- Dates of entry are checked for obvious errors and missing dates of entry added.
- Any blank fields within the data are identified and populated if appropriate.
- All considerations under £20,000 and in excess of £750,000 are checked for accuracy.
- Right to buy sales of council houses to sitting tenants are included in the final RoS statistics, but the quality assurance aims to identify sales of blocks of properties and sales that are not true, market value sales, e.g. transfers of ownership between family members, sales of pro indiviso shares, etc, so that they can be excluded from the extractions.
- Applications which are subsequently cancelled are identified so that they can be excluded from the extractions.
- Foreign currency considerations are identified so that they can be excluded from the extractions.
Summary of house type methodology
Assignation of house type to sale
The house type is identified by using a house type classification system developed by RoS. RoS registers all residential property sales in Scotland, and we initially assign one of the four house types using Ordnance Survey (OS) mapping and address products.
The RoS methodology automatically classifies dwelling types according to the four pre-defined types and uses counts of addresses and neighbouring building polygons to classify each property. This method of classification is detailed in the table below.
| House type | Example | Method of classification | ||
Detached |  | A building with no adjacent addressed building. | ||
Semi - Detached |  | A building with 1 adjacent singly addressed building. | ||
Terraced House |  | A building with one or more singly addressed neighbours. | ||
Multiple Occupancy |  | Any building with 2 or more unique addresses in any single building. | ||
The majority of house types can be identified using the above method to an accuracy rate of 96.4%. When all sale applications have completed the registration process, the house type can be assigned to one of the four house types by this method.
Allocation of unassigned sales
Sales which cannot be identified using this method (approximately 15-30% of the monthly total) are initially categorised as Unassigned. The Unassigned House Types fall into three categories as follows:
- Unknowns - Represents approximately 4% of the monthly total. These are predominately New Build properties without OS address points. The house type will be identified when the property transfer has been fully registered by RoS. Cancelled applications are also included in this category.
- Unaddressed - Represents approximately 1% of the monthly total. These are applications with a RoS Title seedpoint which falls within a building we have identified as 'unaddressable' e.g. garages, sheds and buildings without OS address points. These applications need to be examined manually to determine the accuracy of the sales data and OS address point data.
- None - Represents between 10-25% of the monthly total and is linked to the date of the extraction of the application data. The closer the date of extraction is to the date the application was received by RoS the higher this figure will be. Approximately 80% of the "None" category have not yet passed through the Land Registration process.
All of these unassigned categories will be classified as unassigned with the HPI extractions. For the quarterly, calendar year and property market report statistics, however, a significant proportion (over 95%) of the unassigned categories can be allocated to house type using the following methodology.
All sale applications where RoS can identify the house type are split into the following price bands for each local authority area. An example of this process is shown in the table below. The respective volumes, average price and proportion of property types are calculated from the previous 12 months of sales.
For each price band:
- The proportion of each house type is calculated
- Unassigned sales are then allocated to each house type using these ratios
Allocation using this method transfers the volume and average sale price rather than the actual sale price. For example, in the table above:
- There are 10 unassigned sales within the ”£20,000-£60,000” price band, with an actual sale price total of £570,000
- These would be allocated as 1 Detached, 1 Semi-detached, 4 Terraced and 4 Multi-occupancy properties with an estimated sale price total of £550,000(10 x average price of assigned sales) to maintain the average price of £55,000
- The actual market value in this band is £2,770,000 (40 x £55,000 plus £570,000)
- However, if we multiply the average price by the volume of sales in this band (now 50 x £55,000) the estimated market value would be £2,750,000, a difference of £20,000
Therefore, the estimated market value calculated by multiplying the house type average price by the number of sales will differ from the actual market value of all sales in the period. The difference between the estimated value and the actual value indicates the accuracy of the allocation methodology. Please note that if a large proportion of unassigned sales are present the Annual Change in Average Property Prices for Scotland and for each house type will differ significantly and will over/understate the level of decrease/increase.
A small proportion of sales cannot initially be allocated in this way as these properties have no geographical reference or seed point and therefore cannot be matched to a specific Local Authority. As more information is provided to RoS during the registration process the sale can be assigned a house type. These sales are described as Unallocated in the published statistical tables within the quarterly statistical release.
Revision policy
The revision policies for statistics from Registers of Scotland are outlined below:
- Monthly statistics: each quarter, revisions made to the entire time series, from April 2003 to the latest data period. Data for the months within the quarter are not revised until quarter end
- Quarterly statistics: each quarter, revisions made to the entire time series, from 2003-04 Q1 (April to June 2003) to the latest data period
- Calendar year statistics: each quarter, revisions made to the entire time series, from 2004 to the latest complete calendar year
- Property market report: revised annually, full financial year time series revised. These statistics differ from other RoS statistics as they have been extracted from a live database at a different point in time and are not subject to scheduled revisions throughout the year. The data in this publication, therefore, may not always be the most up-to-date
- Land and property titles by country of origin: annual snapshot of the Land Register so no formal revision process. Annual snapshots from one year to the next are though subject to changes made in the live Land Register database
Revision policy for the UK HPI
Registers of Scotland house price statistics are based on administrative data derived from applications for registration on the Land Register. This data from the Land Register System (LRS) is live and subject to constant change. As a result, for the majority of our statistics the entire time series is revised and users should use the complete time series in the latest available statistics.
As the RoS house price statistics are extracted based on date of registration – the date that the application was received for registration – rather than date of entry, changes in updated transaction volumes are predominantly from the cancellation of sales rather than the inclusion of additional transactions. For this reason the general movement in updated volumes is downward but the overall effect is not sufficiently large enough (or to alter interpretation of the statistics) to put in place formal scheduled revisions.
A provisional label, however, is applied to the latest monthly statistics and latest statistics by house type included in our quarterly, calendar year and property market report statistics.
For the monthly statistics, data for the months within the latest quarter are not revised and labelled as provisional to indicate that they are subject to revision at quarter end along with the entire time series. The monthly statistics are revised every quarter rather than every month as a trade-off against the resources required to update every month compared with the user benefit of receiving the most up-to-date data, given the movement in updated volumes is relatively minor.
The latest house type statistics included in the quarterly, calendar year and property market report statistics are labelled provisional. For the quarterly house type tables, a provisional label is applied to the latest eight quarters. For the calendar year house type tables, a provisional label is applied to the latest two complete calendar years. For the property market report house type tables a provisional label is applied to the latest two financial years. The provisional label is used because of the number of sales that cannot be assigned to a house type. Residential property sales are assigned to one of the house types through a GIS-based classification system developed by RoS. For the latest statistics, a substantial number of sales cannot be assigned to a house type. This is primarily due to time lags between a title being received for registration and the updating of the map base. The long-term average proportion of sales that cannot be unassigned is 3% in a financial year. However, the proportion of unassigned sales increases for the most recent statistics (10% in 2018 -19 and 4% in 2017-18). A provisional label, therefore, is used to caution users when using the latest statistics given the number of sales that remain unassigned.
Other revisions (ad-hoc or unscheduled corrections)
Ad-hoc revisions, for example methodological improvements, or unscheduled corrections owing to identification of errors may occur from time to time. Our approach to such revisions will be in line with the Scottish Government’s revision policy and the Code of Practice for Statistics. Particular attention will be paid to the scale, nature, cause and impact of the changes, and users will be notified as soon as practicable.
For further details on our methodology, get in touch by emailing data@ros.gov.uk.
Footnotes
- HMLR, Registers of Scotland, Land and Property Services Northern Ireland (LPSNI), Office for National Statistics (ONS)↵
- Hedonic regression is widely used for estimating house prices. For a hedonic regression the properties are defined in terms of a set of features or characteristics, each of which contributes to the value of the property. For example, the number of bedrooms or the location of the property will contribute to its value, but these features cannot be priced in isolation. A regression model is used to estimate the value of each of these features from the set of properties sold during a particular period. For example, the model might estimate the effect that every additional bedroom and each different location have in the sale price in a certain month. Then, the price of a particular property can be calculated by combining the values assigned to each of its features.↵
- Data provided from CML.↵
- There are different ways of calculating average prices. These include the arithmetic mean, the geometric mean and the median. The geometric mean involves multiplying the prices together, and then taking the cube (third) root of the total. An arithmetic mean, where all the prices are added up and divided by the number of sales) can result in those areas with the highest priced properties tending to dominate the index, whereas the geometric mean is more equal in its weighting of the change in price of expensive and cheaper properties.↵
- Building polygons are building shapes identified on the OS digital map base.↵
- An “address point” is a point marker used to identify an address within a building on the OS basemap.↵
- A “Title seedpoint” is a point marker used to identify the position of a Land Register Title on the OS basemap.↵