5 Capabilities
The capabilities indicator is defined as: the extent to which the workforce has the right mix of skills. The need for a variety of certain strong skills is vital for the successful operation of any organisation, civil services included. The standards for good governance set out by OPM & CIPFA (2004) include leadership as a core skill, it goes on to list necessary skills as “the ability to scrutinise and challenge information …including skills in financial management and the ability to recognise when outside expert advice is needed”. Fukuyama (2013) acknowledges the importance of educational attainment of civil servants: “another critical measure of capacity is the level of education and professionalisation of government officials”, along with the importance of digital capability: “what level of technical expertise they are required to possess”.
The capabilities indicator is composed of 14 metrics from the OECD’s Programme for the Open Sansnational Assessment of Adult Competencies (referred to as PIAAC from this point onwards), this is an increase of 10 metrics from the 2017 Pilot.
PIAAC is a scientific assessment of competencies in adults, modelled on the OECD’s successful Programme for Open Sansnational Student Assessment (PISA) that measures the competencies of school-aged children around the world.
Data for 25 countries was collected over 2011-12, and data for nine countries was collected over 2014-15. Of these, 31 countries have published microdata available for analysis.
The results from PIAAC are not published in a form that allows for direct import of the relevant data for InCiSE. Instead the data must be calculated from the individual respondent-level microdata published by the OECD. The microdata is analysed to produce results for those defined as currently working in the “public administration” sector of the Open Sansnational Standard Industrial Classification. This is wider than just the civil service and includes other forms of public administration, such as sub-national and local government, but excludes functions such as healthcare, education and transport which may or may not be part of the public sector depending on country.
| Metric | Source | Type | Public sector proxy | Data transformation | Weighting within indicator | Definition of the source metric (e.g. question wording) | ||
|---|---|---|---|---|---|---|---|---|
| In theme (A) | Theme (B) | Total (C=A*B) | ||||||
| Core capability (education attainment and skill levels) | ||||||||
| Literacy capability | PIAAC | Social survey | Yes | None | 25.0% | 25.0% | 6.2% | Percentage of adults scoring high (level 4/5) in literacy. |
| Numeracy capability | PIAAC | Social survey | Yes | None | 25.0% | 25.0% | 6.2% | Percentage of adults scoring high (level 4/5) in numeracy. |
| Problem-solving capability | PIAAC | Social survey | Yes | None | 25.0% | 25.0% | 6.2% | Percentage of adults scoring at level 3 in problem-solving in a technology-rich environment. |
| Educational attainment | PIAAC | Social survey | Yes | None | 25.0% | 25.0% | 6.2% | Percentage of public sector workforce with tertiary education. |
| Use of core skills at work | ||||||||
| ICT use at work [new] | PIAAC | Social survey | Yes | None | 25.0% | 25.0% | 6.2% | Index of use of ICT skills at work (email, internet, spreadsheets, word processing) |
| Numeracy at work [new] | PIAAC | Social survey | Yes | None | 25.0% | 25.0% | 6.2% | Index of use of numeracy skills at work (calculating costs/numbers, charts/graphs, maths/statistics) |
| Reading at work [new] | PIAAC | Social survey | Yes | None | 25.0% | 25.0% | 6.2% | Index of use of reading skills at work (correspondence, media, reference/ academic material, diagrams) |
| Writing at work [new] | PIAAC | Social survey | Yes | None | 25.0% | 25.0% | 6.2% | Index of use of writing skills at work (correspondence, media, reports, filling in forms) |
| Organisational skills | ||||||||
| Influencing at work [new] | PIAAC | Social survey | Yes | None | 33.3% | 25.0% | 8.3% | Index of use of influencing skills at work (giving presentations/advice, instructing others, negotiation) |
| Planning at work [new] | PIAAC | Social survey | Yes | None | 33.3% | 25.0% | 8.3% | Index of use of planning skills at work (planning own and others’ activities, organising own time) |
| Task discretion at work [new] | PIAAC | Social survey | Yes | None | 33.3% | 25.0% | 8.3% | Index of task discretion at work (choice in sequence of tasks, how to do work, rate, working hours) |
| Learning and development | ||||||||
| Learning at work [new] | PIAAC | Social survey | Yes | None | 33.3% | 25.0% | 8.3% | Index of learning in the workplace (learn from colleagues, learn by doing, need to keep up-to-date) |
| Openness to learning [new] | PIAAC | Social survey | Yes | None | 33.3% | 25.0% | 8.3% | Index of readiness to learn (interest in learning, like solving problems, seek out new information) |
| Learning in the past year [new] | PIAAC | Social survey | Yes | None | 33.3% | 25.0% | 8.3% | Percentage who have participated in formal or informal learning for job-related reasons in past 12 months |
| Tables 3.3.A & 3.3.B in the original 2019 publication | ||||||||
5.1 Imputation of missing data
Of the 38 countries selected for the 2019 edition of InCiSE, 10 countries do not have data for the capabilities metrics. As there are countries where data is missing for all metrics the imputation of the capabilities indicator requires a data point from outside the indicator. The 2017 edition of InCiSE used data from the HR Management indicator on applicant skills and whether a country was an EU member. For the 2019 edition, the applicant skills metric from the HR management indicator is retained, but EU membership is removed. One of the metrics within the indicator is the level of tertiary educational attainment. There are a number of sources for estimates of tertiary educational attainment in the general adult population of most countries. Therefore, InCiSE also uses UNESCO data on educational attainment to impute missing data for the capabilities indicator.
5.2 Changes from the 2017 Pilot
The capabilities indicator published in the2019 edition of InCiSE has had a number of changes which improve its quality compared to the data published in the 2017 Pilot. These include additional metrics, change in how data is extracted, updated coding of educational attainment, and changes to imputation. While these do not change the recency of the data, they improve the overall quality of the information provided by the indicator. The OECD intends to update PIAAC every decade, as annual change in the skill level of the adult population does not change rapidly – a general principle in education research is that educational attainment is broadly fixed after young adulthood1. Figure 3.1, shows how the overall proportion of tertiary educational attainment has evolved for different age groups since 1997 in four countries, the average annual change is 0.9 percentage points.
This chart varies slightly from that included in the original published PDF report due to changes in data availability. The original chart included data for the age groups 25-34, 35-44 and 45-54, however the source data at the time of this reproduction (November, 2023) provides data for those aged 25-35 and 55-64.
This chart was Figure 3.1 in the original 2019 publication
5.2.1 Additional metrics
In examining the PIAAC dataset, a number of additional metrics that complement the metrics used in the pilot provide a richer picture of capabilities in the public administration workforce.
The pilot metrics gave a broad overview of employee capability, looking at overall levels of core skills (literacy, numeracy and problem solving) and tertiary educational attainment. The additional metrics complement this by providing for measurement of the use of core skills at work (ICT, numeracy, reading and writing). They also cover more complex skills, including influencing others, planning, and task management. Finally, they also include metrics relating to learning and development — whether individuals learn at work, their overall attitude to learning, and whether they have participated in learning for work-related purposes (either formallyor informally). Together these metrics provide a more detailed picture of the skills and capabilities of the workforce.
Using the public administration industrial sector
The pilot edition of InCiSE used data for all adults currently employed by a public sector organisation. Further investigations of the raw data in PIAAC indicated that there was a sufficient sample size in most countries (n>100) to generate an estimate for the “public administration” industry sector2 and worked for a public sector organisation.
2 Sample sizes for the public administration industry sector (limited to declared public sector workers) range from 83 to 1,562. The minimum and maximum are both noticeable outliers: ignoring these, the sample sizes range from 144-446. The only country with a sample less than 100 (Russian Federation) had similar standard errors to those of other countries and therefore was retained in the data extracted from PIAAC.
There is a considerable difference between countries with regard to whether someone is a public sector worker. This is in part due to the political choices about what is or isn’t delivered by the public sector. For example, in the United Kingdom the vast majority of healthcare workers will be public sector employees, while in the United States the vast majority of healthcare workers will be private sector employees. In contrast, this difference is likely to be much reduced for the “public administration” industry sector, as it will not include sectors such as healthcare, education or competitive market economic sectors. Therefore, while the sample size for the “public administration” subset will be lower, it is likely to be a more appropriate comparator group across countries than using the large “public sector” basis.
Further details on the structure of the activities included in the “public administration” industrial sector can be found in the UN’s registry of Statistical Classifications (UNSD, 2018).
5.2.2 Updated coding of tertiary education
In reviewing the way that results are extracted from PIAAC’s raw data files, an improvement was identified in the way tertiary education is coded. The pilot edition of InCiSE used data from a variable included for legacy comparisons with previous Open Sansnational assessments of adult competencies based on type of institution attended. InCiSE 2019 uses a more accurate method based on the highest level of qualification achieved.
5.2.3 Updating the approach to imputation
In the pilot edition of InCiSE, missing data issues were handled by examining the relationship of the metrics from PIAAC with metrics from the other indicators in InCiSE (as PIAAC is the only data source for the capabilities indicator). The most suitable predictors observed in the dataset were the applicant skills metric from the HR management indicator and whether a country was an EU member. As described above, the imputation for the 2019 edition has changed the methodology to remove the EU membership criteria and include the tertiary education level of the general population in the external imputation data. This provides a closer link to the indicator’s theoretical construct.