Agency Performance Assessment
-- using HUD FY2022 Performance Report as an Example
Performance evaluation is critical in an agency’s self-assessment and public monitoring
process. It yields evidence of a government’s effectiveness, provides direction in budgeting and
people management, and indicates public accountability. (Behn, 2003) This paper evaluates the
FY2022 performance report of the Department of Housing and Urban Development (HUD) as an
example of agency performance assessment, focusing on the alignment, effectiveness, and
significance of the performance indicators as around the strategical goals.
Two strategic goals will be covered with in-depth assessment:
- Strategic Goal #2: Ensure Access to and Increase the Production of Affordable Housing
- Strategic Goal #4: Advance Sustainable Communities
1. Strategic Goal #2
Strategic Goal #2 has two objectives about housing supply and rental assistance. These two
objectives directly address the strategic goal.
1.1 Alignment of indicators with the goal and objective
All the indicators under housing supply address the numbers of housing produced, and the
programs/schemes are directly related to financial assistance. First, all the indicators have
focused on the numbers of different types of housing units. The numbers of units provided to the
tenants are directly related to HUD’s capacity in housing supply. A larger number of housing
supply would benefit more target tenants who need assistance in owning or renting a place to
live. Second, almost all indicators are measurements about the financial assistance HUD
provides. Financial assistance serves as immediate paths to housing accessibility. The indicators
are also about a good combination of the housing programs HUD provided – for example, MAP
and Risk-Sharing construction/rehabilitation programs.
Similarly, the indicators under rental assistance also focus on the numbers and percentages.
Three indicators – the occupancy rates, the utilization rates of housing vouchers, and number of
families served by rental assistance are closely related to the performance of rental assistance
programs. The other two indicators on housing physical inspections for public housing and non-
compliant multifamily properties further address the quality of living, safety, and health,
therefore would go along with the goal by adding additional spectrum of the impact from
affordable housing, which is in alignment with the strategic goal.
1.2 Effectiveness and meaningfulness of the indicators
Overall, HUD presents the set of housing programs in the report and has concrete records to
prove their performance. The indicators are relevant and trustworthy. Take the below indicator as
an example, the number of market units has track records throughout the years dated back to
FY18. The type of market-rates units is clearly explained as defined by the legal clause. The
indicators are trustworthy with constant source, precise definition, and the pattern of evolvement
looks real. The indicator also closely addresses the issue of housing supply. The indicator is able
to manifest the performance challenge: both the market-rate sector and the affordable sector are
not performing impressively during the COVID years; only the affordable sector met the target.
Similar observation applies to other indicators under this objective, including the number of
housing units protected by FHA multifamily mortgage insurance, the number of units under
rental assistance demonstration for FHA, the number of units under the HOME partnership
program, etc. Some other indicators show some extent of growth in FY22, which seem authentic
and convincing. It’s worth noting that FY22’s actual supply outperformed its target; however, the
target is way below the historical targets. By comparing to the historical supply, FY22
underperforms in the housing supply under HOME partnership programs. However, there is no
further explanation here about the rationale behind target setting. This seems to be alluded to in
the previous context from reading the report, that HUD has been facing the challenge with
affordable housing supply – the annual target adjustment indicates the acknowledgement of the
hardship.
The performance report also formatted indicators followed by a table detailing the
description of the indicator, related legal clause, data source, measurement unit, method,
evaluation on data quality, and verification. This is an all-round self-evaluation mechanism. With
sufficient information provided, this mechanism could improve evidence-building and thus
enhance the convincingness and effectiveness of indicators.
Although the set of performance indicators highlights the properties and programs HUD
provides, in terms of the depth and effectiveness, it would have taken a step forward to measure
the substantial impact. With the increase in housing supply numbers, do all the programs help
improve the situation for most tenants who need assistance? Have the tenants obtained long-term
housing security? To answer these questions, the measure may need to go beyond the simple
snapshots of current housing supplies – it requires accumulative records of the measure over a
period, as well as multiple tracks of indicators to manifest the substantial changes. In this way,
the indicators would reveal whether HUD’s policy instruments generate positive impact on the
ongoing housing crisis, and whether the structural disparities of housing supply and accessibility
have been alleviated.
Similarly, on rental assistance, indicators of occupation rates, housing voucher utilization,
and number of families served provide a snapshot of the current situation. Other aspects
mentioned in the evidence section have not been developed into indicators. Such aspects include
providing assistance with poorly performing multifamily tenants, improving the communities’
accessibility and connectivity to major locations, etc. Indicators would be further developed and
enriched to measure the advancement in the quality of living. Again, in order to unveil the
substantial impact on the overall situation of the population who needs rental assistance, more
statistics on the overall percentage increase, and details about specific aspects of long-term
benefits would be helpful.
2. Strategic Goal #4
Goal #4 is newly drafted since 2021 when Biden addressed environment resilience and
justice in his President Executive Order, therefore, the indicators only show records starting
FY21/22. Goal #4 address two pillars: climate resilience and environmental equity.
2.1 Alignment of indicators with the goal and objective
In general, this part has indicators rooted in the national greenhouse gas emission goals,
green building standards, disaster resilience funding programs. Despite being relevant, the set of
indicators has yet touched on the large major aspects in the spirit of environmental justice – a
well-round set of measurements is yet to be developed to match the broad goal.
The indicators under climate resilience are relevant in general, gathering evidence including
the action plan accomplishment, disaster resilience, and energy efficiency in the housing
programs. Indicators under environmental equity, however, have been framed with a focus on
home safety, along with the percentage of the action plan accomplishment, and remediation
investment. With the advocacy of justice for underserved populations, in particular low-income
households and communities of color, existing indicators have yet to address such elements.
2.2 Effectiveness and meaningfulness of the indicators
As compared to affordable housing, indicators under this goal are not solid, supported by
vague descriptions of the methodologies. Such ambiguity might be due to the lack of evidence –
thinking of the process and time required to build up environmental track records. It’s also under
the impression that the measurements are put together in a rushed manner.
For example, the GHG emission targets, as of 32%, 80%, 92% seem ambitious, especially for
FY23. This big leap forward might align with the national GHG goals, but it is questionable on
the feasibility side. Environmental justice is even harder to substantiate and measure. Contrast to
the affordable housing objectives, in the sub-metric table (below), description comes in vague
terms, data source is simply presented as “multiple”, “research and technical contracts”, and
“identified programs” in the first indicator – language like this fails to provide information about
data source and methodolgy.
The report puts two major indicators: the number of at-risk housing units (lead-safe), the
percentage in terms of achieved Climate Action Plan, and federal dollar spent in remediation.
The indicators are vague and questionable. At-risk housing units and lead safety are more
relevant to safety issues and may be more relevant to the Goal 1 under housing justice. The
percentage of achievement in the climate action plan has been used both to measure climate
resilience and environmental justice. However, using percentage to measure a policy plan on
climate issues sounds questionable. There is no elaboration of methodologies illustrating how
such achievement of multifaceted policy instruments come to be normalized. The report does
mention “counts” of action items – however the measurement could be more reliable if the
methodology weigh the items based on policy priorities, and better elaborate the definition of
“completeness” of action items.
As contrast to the affordable housing objectives, the language with environmental targets is
vague with many broad terms. This is an example of a common tactic used by an agency with
performance objectives - government officials lack motivation in providing clearly defined
performance targets, “because objectives are open to constant interpretation and reinterpretation
at every stage of the policy process”. Ambiguous objectives leave wiggle room for further
interpretation therefore help to alleviate public criticism and the chance of failing the objective.
(Behn, 2003; Joyce, 1997) However, such efforts alleviate the effectiveness of an objective in
providing pointers of the policy progress, problem improvement, and the overall achievement the
agency performed. (Behn, 2003)
3. Conclusion
Under Goal #2, the indicators and measures align with the objectives. The indicators
successfully depict the performance patterns in recent years: with steady affordable housing
supply, decreased market-sector units and all other housing programs, HUD has slowed down in
affordable housing supply, as negatively impacted by COVID.
Goal #2 has been implemented and evaluated in a pragmatic manner with evidence of
specific numbers and facts HUD achieved. The evidence mostly shows the snapshot of one
year’s achievement – indicators could be further developed to unveil the substantial impact on
the low-incomed and underserved groups’ living condition. Such impacts might be tactically
hidden considering the hardship HUD has been going through with COVID in terms of housing
supply. It could be an even bigger challenge for HUD to alleviate the structural disparities in
housing accessibility for lower-income population and underserved population.
Goal #4, as comparison, appears to have spent more brushstrokes on the political narrative
than the actual measurable incidents. Some of the indicators under Goal #4 are hard to unpack
based on the information provided in the performance report.
The environmental indicators are underdeveloped, although HUD frames environmental
justice around housing safety and environmental hazard, more indicators should be formulated to
better reveal the big picture of environmental justice – to showcase what HUD has done, and to
what extent the efforts has changed the situation for the underserved communities. By using
vague language, the agency can leave wiggle room for further interpretation about climate and
environment agenda, and less prone to public criticism if policy results are not concrete or
deviate from the goal.
Reference
Behn, Bob. (2003). Why Measure Performance? Different Purposes Require Different Measures.
Public Administration Review 63(5): 586-606.5