Assuming that urban planners (our target group) are likely to seek tools for their daily work using a search engine rather than via scientific papers, we decided to use the world’s most popular such engine for our study, namely Google search (Statcounter 2021).

A systematic Google search was conducted from September to October 2020. German and English search terms were used to locate national and international tools. During each search, the individual results were screened to see if a relevant tool was listed on the indicated web page. Here, the first step was to check for the terms “tool” or “toolbox”. Then we assessed whether the “tool” dealt with heat stress and respective adaptation measures in the city. While this was true of some tools, others were described as being designed for other fields such as “energy” or “rainwater management”; in such cases, they were excluded from further analysis. If a tool were merely referenced, the provided link (or Google search) was used to search for the tool until the correct website was found. If several tools were listed on a resulting website or if reference were made to further tools, these were all subject to an immediate assessment and, where relevant, included in the analysis. Due to the high number of results, with frequent duplications as well as irrelevant information, the search process was declared finished for each search term after Google issued the message: “In order to show you the most relevant results, we have omitted some entries very similar to the […] already displayed. If you like, you can repeat the search with the omitted results included” (see Table 1).

In addition, a literature search was conducted on articles published on the ISI Web of Science website using the search terms “heat resilient city” AND “tool” and “heat adaptation city” AND “tool”. Neither of these searches resulted in additional tools for analysis.

The next step was to determine exclusion criteria to refine the search results and locate the tools relevant to the topic. If one of these criteria were found to apply to a tool, it was excluded from further analysis. The exclusion criteria are listed and briefly explained in Table 2 (note: the ordering has no significance).

After determining the exclusion criteria, it was necessary to define assessment criteria. These were used to evaluate the performance of the selected tools. Clearly, the criteria had to reflect those features that a tool must possess for everyday use in planning heat adaptation measures at city, district or site level. To this end, we chose the list of criteria for a useful planning tool suggested by two focus groups of 15 participants active in several fields of urban planning and decision-making, as recorded in the study by Van Oijstaeijen et al. (2020). However, due to the divergent objectives of the tools discussed in Van Oijstaeijen et al. (2020) (values of green infrastructure in general) and our research (focus on heat stress), some of these criteria were modified, omitted or replaced. The assessment criteria applied here are detailed in Table 3. Further, the necessary data input and resulting output for each tool was considdered and summarised and are supplied in the supplementary material (Suppl. material 2). The complexity of data gathering is reflected in the expertise-rating of each tool considering whether expert knowledge is needed for data gathering or not.

A 4-point scale was used to evaluate the performance of the selected tools according to each assessment criteria. These ratings were: “++” for very suitable, “+” for suitable, “0” for acceptable and “-” for unsuitable as a decision-support tool in the sense of this study. For additional information gathering on the selected tools, we consulted the respective manuals, websites, press releases, video tutorials and scientific publications on the methods or case studies.

Based on the analysis and comparison of the different tools, we then identified optimisation potentials, as well as further research gaps. These will be detailed in the discussion section.

All tools (apart from GREENPASS) are available free of charge, making them suitable and affordable for quick assessment (see Fig. 3). Tools that are quick to use are often designed in such a way that they also require little expertise. In fact, all tools, except GREENPASS, can be run without prior experience using default values and benefit transfer methods for a general assessment and exploration of measures and/or their effectiveness (see Fig. 3). These tools have the advantage of easy application to areas where data are scarce. Nonetheless, Van Oijstaeijen et al. (2020) state that such assessment tools should be used with care, claiming that users must ideally possess an extensive understanding of the underlying methods and limitations. The tools Betroffenheitswizard and Future Cities Adaptation Compass were designed as comprehensive screening tools, producing very general quantified values (see Fig. 3). While the effectiveness of measures is also addressed in Future Cities Adaptation Compass, the results are categories rather than concrete values. These two tools are primarily designed to identify options for municipal action with regard to climate change adaptation. The low spatial resolution of the CLARITY tool is also only designed for general screening, here with a focus on thermal stress (see Fig. 3). As this tool offers concrete as well as extensive quantification of vulnerability and adaptation strategies, it is suitable for a quick, if rough, assessment of the urban heat situation.

As the tools were mostly developed for specific areas, they have extremely limited transferability (e.g. UHI-DS). Those tools designed for a wider applicability usually employ benefit transfer or spatial proxy methods, giving generalised results and reducing the reliability of outcomes (Eigenbrod et al. 2010). Some tools attempt to improve the accuracy of their outputs by including some local data in the form of geodata or socio-economic data. However, such local datasets are usually very general, without a clear reference to the on-site microclimatic situation. Only three tools can accommodate local geo- or climate data. GREENPASS directly integrates climate data (based on landcover and land-use types) in the urban climate model (ENVI-met). The CLARITY Advanced Screening tool uses local open-source geo- and climate data for certain cities at a spatial resolution of 500 m² (which is too imprecise for site level). While developers can adapt the tool to other cities or to a more detailed resolution with the right datasets, this requires additional time and expertise (see Fig. 3). The Adaptation Support Tool V1 (AST 1) can be adapted to local-specific climate data with the involvement of the developers in a rather time-consuming process.

With the exception of CLARITY, all tools were designed to investigate “heat stress” seen as the impact on the human well-being and health (along with some other issues). However, only three tools are able to assess the microclimatic situation, namely GREENPASS, Adaptation Support Tool V2 and the UHI-DS tool, AST 2 and UHI-DS, but are not transferable to other cases. In Fig. 3, we also see that socioeconomic vulnerability is only investigated by the four tools operating at larger scales. Furthermore, CLARITY is the only tool to work with actual geodata on the local population. As inputs, Betroffenheitswizard and the Adaptation Compass require rather crude, non-specific data. CLARITY and the C40 tool are the only tools to use socio-economic data to analyse the impact of measures. Both tools completely neglect the localisation of, for example, social institutions of particularly vulnerable groups, such as kindergartens or nursing homes. CLARITY focuses solely on population density. While C40 considers age structure, this is not for the purpose of localisation, but rather to calculate the likely spread of heat-related disease. The gathering of socio-economic data may be more important for the localisation of particularly vulnerable areas at the city and district level than at the site level.

On the other hand, almost all tools attempt to assess vulnerability to thermal stress, albeit in very different ways. One exception here is AST 1, which does not capture vulnerability via meteorological variables or socio-economical values. Like the C40 tool, it cannot represent the actual climatic situation and, thus, only analyses the effectiveness of measures in comparison to a worst-case scenario. While the INKAS tool also only considers the effect of measures, the user has the option to estimate the actual situation via spatial analysis diagrams. Apart from Adaptation Compass and Betroffenheitswizard, all tools allow the planning status to be altered, which is very useful for planning decisions and to derive optimal solutions.

In general, it is helpful for planners if comprehensive outputs are offered, especially monetary values, in order to convince investors and decision-makers of the importance of implementing measures (Van Oijstaeijen et al. 2020). Except for Adaptation Compass and the Betroffenheitswizard, all tools use at least biophysical metrics. INKAS and the UHI-DS tool can only calculate temperature indices. While this may be sufficient to determine the thermal impact of a measure for planning purposes, such data are scarcely suitable to persuade investors or other stakeholders. For this reason, it is generally necessary to calculate monetary values in addition to biophysical metrics, as well as (where applicable) socio-economic data. These are all provided by five of the tools (see Fig. 3). GREENPASS, AST 1 and AST 2 are able to quantify investment and maintenance costs. Potential cost reductions – presumably particularly decisive for planning processes (e.g. reduction in hospital costs) – are provided as outputs in the tools GREENPASS, CLARITY, C40 and AST 2. It should be noted that Fig. 3 does not indicate the scope of measures that tools can accommodate. While the measures in GREENPASS are presumably freely selectable, some tools are extremely limited in the measures that can be quantified. For instance, the UHI-DS tool can only analyse the impact of a few specified measures; the C40 tool can evaluate only four rather generalised types of measure, all of which must be of a certain size; and the INKAS tool also considers only a fairly limited list of measures. In general, our analysis showed that almost no tool can be applied to small individual measures on small areas, such as a single tree, even though these can have a large local climatic effect (Westermann et al. 2021). While the effectiveness of such measures is probably difficult to evaluate for an entire district, they can clearly boost the well-being of local residents. AST 2 is the most innovative in this sense, as it can take account of small individual measures and their effect via a PET map (Physiological equivalent temperature, Höppe 1999).

In almost all cases, the descriptions of the tools stated that they were developed especially for urban planners to support the decision-making process. Usually, (several) research institutions were involved in the development over a lengthy period. Consequently, the tools all show a high degree of sophistication. Unfortunately, most tools do not address the microclimate, which is so crucial for analysing the impact of heat stress on the well-being and health of urban residents. The accompanying documentation to the tools repeatedly points out that they cannot replace detailed urban climate simulations based on local data. Here the question arises whether detailed studies for smaller project areas are, in fact, necessary. Van de Ven et al. (2016), for instance, describe from experience that measures are often implemented in practice, based solely on their expected effect. Van Oijstaeijen et al. (2020) also point out that municipalities frequently lack the time and resources to conduct intensive evaluation processes at project level. The tools assessed here are generally intended for the early planning phases, when preliminary generalised values and outputs can help shape the later planning stages. However, there remains a risk of over- or underestimating the impact of various measures, leading to their (perhaps unjustified) acceptance or rejection.

In the following, our analysis results are used to pinpoint those elements which tools must possess in order to be useful to planners.

Currently, many tools neglect the microclimate. While temperature indices can be found in almost all examined tools, these are generally only used to roughly record the overall situation and are not localised, for example, via geodata. Clearly, the use of geodata can strengthen the scientific basis of results and save time in data gathering (Van Oijstaeijen et al. 2020). The use of local microclimate data by tools will certainly ensure the quick transferability of results to similar (climatic) conditions without having to undertake extensive data collection and simulations, when needed data are provided by the tool itself. However, we acknowledge that local climate data are often hard to obtain and availability can considerably differ depending on the area studied. This could hinder the user to take advantage of a tool. For a first evaluation of the overall situation, easy tools with a low demand of input data might also be very helpful, especially for users with less expertise. The CLARITY tool has adopted a good approach here by linking open-source datasets, although the spatial resolution is still too low to be able to map microclimatic conditions. In the course of the development of AST 2, a PET map covering the whole of The Netherlands was created to locate areas of heat stress. This comprehensive data can potentially help to express the biophysical effects of adaptation measures in a particularly meaningful way for decision-making processes. Further research is required to design ways of representing and assessing the microclimate and heat stress situation, based on simple, generalising features and conditions. Such approaches could certainly bring useful features to planning tools.

The consideration of small site scales is particularly useful when planning heat adaptation measures. Many of the tools presented here, however, operate on other scales, such as city or district level (see Fig. 3). When the micro-scale climate situation is neglected (as in most of the examined tools), only those heat reduction measures with a measurable effect on the entire neighbourhood can be modelled. Consequently, very small-scale or specific measures are ignored by tools, such as the greening of bus stops and the introduction of individual trees. Measures that can significantly improve the quality of life by reducing heat stress, but do not achieve a measurable effect for the whole neighbourhood, are virtually excluded from the outset. Here, further research is needed to formulate requirements for future tools to ensure a stronger focus on site scales. Often there is a lack of relevant studies and knowledge, as shown in the review by Brzoska and Spāģe (2020) of the evaluation of urban ES.

In addition, the recommended measures in the tools are presented in a generalised way. This can foster user engagement with the measures and, thus, constitute a learning effect. However, it can also lead to the user feeling overwhelmed, as it becomes difficult to determine which measure makes the most sense and which is most effective for any particular type of urban structure. In some tools, the measures are insufficiently described. The user, therefore, has to inform herself/himself extensively, a step which takes time and could lead to a loss of interest and information. Therefore, it would be better if highly specific measures were proposed. Almost every tool makes recommendations by linking together diverse features using filters or scores. Often, for example, the measures are linked to the potential for heat reduction or the building type. Combinations of different filters are more likely to be implemented in filterable catalogues of measures. The AST is the only tool that provides more precise recommendations by scoring a pre-selection of different characteristics of the environment (e.g. slope, soil availability, soil type), urban structure types (development type, fallow land, sports field, extensive green space, grey sealed area etc.) and objectives (e.g. heat reduction, drought reduction, flood safety). More specific results will be given if there is no multiple selection of characteristics for divergent areas, but in fact. only one area is considered in isolation and the information is adjusted to that one site. This could possibly be concretised in the AST or in a novel map-based tool by the user assigning concrete properties to individual sites in an area in a standardised way, enabling the recommendation of targeted measures. Certainly, this would require a considerable volume of input data for large areas; for small project areas, however, the time required would probably be fairly modest, thereby offering the user more concrete decision support. It makes sense to locate or link the advantages and disadvantages, the execution variants and examples briefly and precisely in the tool itself. A very detailed tool for measure recommendations could be created if the user inputs the most precise possible specifications of urban structure types and their characteristics, such as area size, degree of sealing, vegetation or similar.

Furthermore, monetary values could be expanded to support the decision-making process. Here, the tools apply different methods to quantify cost reductions in the health sector or the estimated costs of measures, for example. Since these values, in particular, could sway the decisions of investors, it makes sense to integrate as many such values as possible. However, most tools currently offer only a few such values.

Co-benefits provide further arguments for the implementation of green- and blue infrastructure in planning processes, yet these are hardly reflected in the investigated tools. Therefore, co-benefits could be included in a new tool, alongside socio-economic values, which should be considered more extensively. Our analysis also determined that the tools only address daytime heat stress while ignoring the nocturnal heat load, which is particularly important for physical and mental recovery (Baumüller 2008). Heat stress at night is only modelled in GREENPASS via the PET maps. This deficit could be remedied in a newly-designed tool. However, measures for heat stress at night may partly conflict with those intended to reduce daytime heat stress, for example, trees providing shade during the day can reduce ventilation and cooling at night (see Henninger and Weber 2019). It could prove challenging to reconcile these features in a tool designed to be as simple as possible.

Most of the assessed tools were developed in Germany and Europe; this can be attributed to the specific search terms, as well as the geographical location of our search. As previously mentioned, we made use of Google search to locate the investigated tools. This method was chosen as it reflects the likely approach of a typical planning office faced with limited staff and time. However, the use of the Google search engine raises a few uncertainties that should be pointed out. On the one hand, Google weights the search results on the basis of ranking systems and search algorithms (Google 2022). Further, the search is also influenced, for example, by the user’s location, the topicality of the web pages and other factors. It is, therefore, questionable whether the same results would be displayed if the method were repeated later at a different location. It is also uncertain whether all relevant web pages are actually displayed by the search engine. In the applied method, the search for the respective search term was declared to have ended when – according to Google – all of the most relevant results were displayed. This step is determined by Google’s search algorithms. However, since the number of non-relevant search results (according to Google) appeared far too extensive for any manual analysis (see Table 1), the most relevant Google results were taken as sufficient. Ultimately, the search results will closely reflect the chosen search terms. The individual search results and, where applicable, their subpages were each screened for relevant tools. Although this procedure was carried out very conscientiously, it cannot be ruled out that some relevant information was missed, for example, due to non-functioning links. Regarding the search method and its uncertainties, the selected tool list does not claim to be exhaustive. The platform Web of Science was consulted for completeness in order to validate our search method. In addition, it became clear during the analysis that scientific articles simply do not exist for all tools; therefore, the fact that websites and manuals must necessarily serve as sources of information, reinforces the validity of the simple Google search.

Regarding the analysis of the selected tools, it should be pointed out that a large number of the assessment criteria were drawn from those identified by Van Oijstaeijen et al. (2020). In the future, however, the process of criteria selection should ideally involve potential users of the planning tools. Additionally, the assessment criteria had to be greatly simplified to enable a meaningful evaluation. There is some interdependency between individual criteria, such as expertise and time requirement, which needs to be taken into account when interpreting the results. Another limitation is that the evaluation of the tool performance (see Fig. 4) was solely conducted by the authors of this study. A larger-scale survey including potential users of the tools (planners) would certainly increase the reliability of the assessment of tool performance. This should be considered in future work. In addition, we must point out that GREENPASS could only be evaluated to a limited extent as the relevant resources and the tool itself were not freely available.

In general, the term “tool” is all encompassing, covering diverse planning tools and instruments that can be either digital or analogue (guidelines, maps, brochures etc.) and many results were excluded. For this study, therefore, we had to give a closer definition of the meaning of “tool” (see Introduction). This reduced our sample to a small group of tools (eight in total, see Fig. 2) that could be considered for use in everyday planning.

This component will assess the effectiveness of different adaptation measures in reducing urban heat. Their impact will be evaluated with regard to human heat stress, which encompasses short- and longwave radiation, air temperature and humidity, wind speed and direction. Here we make use of the Universal Thermal Climate Index (UTCI, Bröde et al. 2011, Jendritzky et al. 2011, Błażejczyk et al. 2018), currently one of the most frequently applied indices to measure heat stress. The tool component “open space thermal comfort” is based on urban microclimate simulations created by the model ENVI-met 3.1 (Bruse and Fleer 1998, Bruse 1999). Here, areas of size 12 x 12 m representing certain initial conditions and areas representing certain adaptation measures (e.g. increasing the amount of trees) were extracted from real-world simulations (Ziemann et al. 2019). The corresponding UTCI values were determined from these simulation results (Westermann et al. 2021). From a pre-selection of areas obtained in this way, the user can first select the one which best matches the target area whose thermal comfort is to be improved. In a next step, the user can select an area best matching the adaptation measure to heat s/he wishes to apply. Both chosen areas are visualised and a verbal description supplied. The evaluation of the effectiveness of the measure is based on the difference in UTCI between the two chosen areas (i.e. with and without the adaptation measure). As the tool output, the user is informed about the effectiveness of the chosen measure by means of a traffic-light rating system for three different hot summer days (mid-June, mid-July and mid-August) and three different times of the day (the evening, early morning and average conditions over mid-day). This is complemented by a short statement and some further details, where necessary (e.g. if the adaptation measure has little thermal effect, but may be still useful for other reasons). In addition, the difference in UTCI is reported, as well as the stages of thermal stress for both chosen areas. This concept will be implemented to evaluate the effect of individual measures of limited scope, as well as to assess the effect of heat adaptation measures on city streets (e.g. the planting of street trees). Generally, it will be also possible to assess the effect of measures which could potentially increase the heat stress suffered by residents (e.g. when trees are cut down). It is important to note that the “open space thermal comfort” component will provide some preliminary quantitative results that are much more useful than the mere qualitative statement “Urban green is good”, but which cannot replace detailed simulations of the urban climate in a special situation and for a real or planned district. The tool will be supplied with a manual, which comes in two versions (one shorter and one providing more details).

When the tool is ready, this component will probably end up being a subcomponent of the “open space thermal comfort”. It will provide the user with a description and evaluation of further co-benefits provided by certain heat adaptation measures and, thus, create a more comprehensive view of their diverse impact (here: on ecosystem services).

The ES assessment of the pre-defined model areas (which are also used for the open space climate simulation) follows the approach developed by Brzoska et al. (2021). This was transferred into practical implementation after being developed in the first phase of the HRC project. The approach applies a multi-criteria decision-making process to derive results on the provision of ES in a certain area, specifically the services of “passive recreation” and “nature experience”. In addition, other co-benefits, such as carbon storage and biodiversity, are identified and their individual function briefly described. In this way, the tool will offer qualitative statements on their function as impacted by the chosen measures, as well as a more detailed assessment of the supply of the ES “passive recreation” and “nature experience”. The results of the ES assessment could serve as an important additional aid to the decision-making processes for the potential realisation of certain measures.

The second main component of the tool assesses the impact of adaptation measures on indoor overheating in residential buildings. In a first step, the user will be able to assess the current indoor thermal state of a room by providing information on the building type, the exposure of windows and location of the room within the building, for example. Based on these data, the risk of overheating is visualised by means of a five-level traffic-light system (from red for very high risk to green for minimum risk). After defining the initial state of the room, the user can choose different heat adaptation measures, such as solar shading devices or green roofs. Application of the selected measure to the initial room conditions may change the intensity of overheating, indicated by a shift in the traffic light, for example, from red to yellow or green. The assessment of the initial risk of overheating and the adapted risk are both based on simulations of building performance and indoor comfort monitoring for different residential buildings conducted in the HeatResilientCity I and II projects, as well as other projects (Kunze et al. 2020, Schünemann et al. 2020b, Schünemann et al. 2020a, Ortlepp et al. 2021, Schünemann et al. 2021a, Schünemann et al. 2021b, Schiela and Schünemann 2021). It is important to note that the indoor climate component of the tool offers a qualitative assessment of the chosen room by means of the traffic-light system without giving any quantitative estimates of the effect of the heat adaptation measures. This limitation is due to the large number of parameters that influence the indoor overheating risk. For a more detailed, quantitative evaluation of the impact of heat adaptation measures for a specific room or building, we suggest the use of building performance simulations, where possible, for the entire building in order to take account of internal air exchange and heat transmission.