The U.S. Government Accountability Office’s Technology Assessment Design Handbook [PDF] was released in December, 2019. It identifies tools and approaches to use in the design of robust and rigorous technology assessments (TAs). The handbook underscores the importance of TA design, outlines the process of designing TAs, and describes approaches for mitigating selected TA design and implementation challenges . While the primary audience of this handbook was GAO staff, other organizations engaged or interested in TAs will find portions of this handbook useful.
GAO identifies three general design phases, shown in the figure below. The handbook also highlights seven cross-cutting considerations for designing TAs: the iterative nature of TA design, the requester’s interests, resources, independence, stakeholder engagement, potential challenges, and communication. In addition, the handbook provides a high-level process for developing policy options, as a tool for analyzing and articulating what a policymaker could do in the context of a given technology and policy goal. Steps in developing policy options include, as applicable: formulating initial policy options to consider; gathering evidence, determining relevant dimensions to analyze, and analyzing the policy options; and presenting the results of the policy analysis.
Summary of Key Phases of Technology Assessment Design
Through it’s work advising Congress and federal Executive Branch agencies, GAO found that TAs have and can use a variety of design approaches and methods. The handbook provides TA design and methodology examples, including related to objectives commonly found in GAO TAs, such as: describe a technology, assess opportunities and challenges of a technology, and assess policy considerations. One example provided is: some GAO TAs include an objective related to describing the status and feasibility of a technology, which GAO teams have done by using methodologies such as expert panels, interviews, literature and document reviews, site visits, and determining the Technology Readiness Level.
Also included in the handbook are examples of TA design and implementation challenges we found, along with possible mitigation strategies. We identified four general categories of challenges, including: (1) ensuring TA products are useful for Congress and others; (2) determining policy goals and measuring impact; (3) researching and communicating complicated issues; and (4) engaging all relevant stakeholders. An example of a potential mitigation strategy to the specific challenge of writing simply and clearly about technical subjects includes: allowing sufficient amount of time for writing, including reviewing and revising writing.