Research Specialist, National Council on Disability. (The views expressed in this article do not purport to represent the views of the National Council on Disability.)

The Americans with Disabilities Act (ADA) aims to strike a fair balance between the rights of people with disabilities and the legitimate concerns of the business community and other entities covered by the legislation. To do so, the conceptual framework and language of ADA emphasize empowerment, independence, and inclusion of individuals with disabilities in all aspects of community life, but also "reasonable accommodation" and "undue hardship". By design, ADA is expected to interface with a host of existing and future laws.

In passing ADA (Public Law 101-336) in 1990, Congress recognized that millions of Americans with disabilities have been isolated and segregated; faced restrictions and limitations; occupied an inferior status; and have been seriously disadvantaged. Congress determined that unacceptable discrimination occurs when individuals with disabilities' access to goods and services of businesses and federal and state government agencies is limited. To remedy these problems, ADA established a clear and comprehensive prohibition of discrimination on the basis of disability. ADA defines disability as (a) a physical or mental impairment that substantially limits one or more major life activities; (b) a record of such an impairment; or (c) being regarded as having such an impairment.

ADA is the most comprehensive federal civil-rights statute protecting people with disabilities. It affects access to employment (Title I); access to state and local government programs and services, and to places of public accommodation such as businesses, transportation, and non-profit service providers (Titles II and III); and telecommunications (Title IV). Title V contains miscellaneous provisions, such as technical assistance, coverage related to Congress and its agencies, and alternative dispute resolution.

With ADA, a new paradigm of disability was firmly established in law and policy. Moving away from the medical model that traditionally formed the foundation of disability policy, this paradigm offers a civil rights orientation and focuses on societal barriers to full participation rather than individual's functional impairments. The disability community has embraced ADA as its "Declaration of Independence".

Have people's lives improved? How effective has the ADA been in combating discrimination? What impact has the ADA wrought in America? To approach answers to those questions requires a general discussion about assessing civil rights laws.

Policy-makers looking to assess the impact of civil rights laws face challenges inherent to the very nature of societal change. Outcomes of anti-discrimination laws for the intended beneficiaries often occur a long way downstream and may not take the form anticipated. Outcomes depend on society's responsiveness to a range of identifiable and unidentifiable factors. This makes it difficult to: a) identify and attribute specific outcomes to specific components of a civil rights law and their related activities and programs; and b) aggregate and compare results across multiple levels of governmental activities and initiatives during the course of the law's implementation.

Policy-makers are under pressure to measure and be accountable for the rights law's impacts. Nevertheless, they should not rush into evaluation using inadequate methods that assume a direct cause-and-effect relationship between a primary problem (e.g., discrimination or bias against people with disabilities), a secondary problem (e.g., high unemployment among people with disabilities), and a given "solution" (e.g., passage of ADA). As noted, societal change is a complex process; a law's implementing programs cannot be isolated from the many institutions (e.g. government departments, community decision-makers, businesses and industries) with which it will interact, nor from broader social, political, cultural, economic, and historical factors that influence it.

The challenge for researchers is to simplify the overall approach enough to plan and implement an evaluation, while methodologically accounting for the contextual reality of the law. Rather than looking for neat linear cause-and-effect relationships, it is prudent to think in terms of a "chain of influence." Doing so permits one to identify characteristics and effects of the law at key points in its implementation -- inputs, activities, outputs, and outcomes -- thereby producing a framework for data collection and analysis across stages of implementation.

Inputs are resources a program uses to achieve a law's legislated objectives, e.g., staff, facilities, equipment, curricula, and money. A program uses inputs to support activities. Activities are what a program does with its inputs --the services it provides-- to fulfill its mission, e.g., complaint handling, compliance monitoring, public education about reasonable accommodations, and technical assistance on accessibility. Activities result in outputs (or "units of service") such as the number of complaints handled, cases closed, technical assistance brochures distributed, or participants served. A program's outputs should produce desired outcomes for the program's participants during or after their involvement with the program. Outcomes may consist of knowledge, skills, attitudes, values, behaviors, living conditions, or social status. Examples include greater knowledge of accessibility strategies, getting a job, homeownership, improved health, or enhanced financial security.

To determine the "results" of a civil rights law, policymakers examining points in its implementation could investigate: (a) costs, e.g., number of federal staff assigned to handle ADA complaints, as an "input"; (b) the number of complaints processed annually, as an "activity"; (c) compliance and/or enforcement, such as the number of settlements, as an "output"; and (d) lifestyle "outcomes" – e.g., homeownership, post-secondary education, or employment---as a result of the law. How to do all that must, however, be considered within a comprehensive evaluation framework.

Anti-discrimination laws typically do not incorporate evaluation or assessment provisions. The sense of Congress appears to be that there is substantial evidence of discrimination against a class of people that has been occurring for a long period of time; it has caused them harm, and is likely to continue unabated unless Congress intervenes.

My analysis of the data bases and research conducted on the ADA (see "Side Bar" below) reveals two broad challenges for judging the law's impact: (a) the wide distribution of administration and enforcement of the ADA, and (b) definitional, measurement, and scope-of-research issues.

Responsibility for administering ADA is dispersed among federal executive departments and agencies as well as certain units in Congress and other legislative branch entities. A major function of the administering bodies is complaint-resolution, through voluntary compliance whenever possible, a process that includes assessing incoming complaints, mediation and legal action as a last resort.

One consequence of ADA's dispersed administration is that few data exist on administrative resources employed in its implementation. There has been no systematic attempt to collect data from all administering departments and agencies; nor does this seem to be of much interest to U.S. officials, mainly because administrative cost is regarded as only a tiny fraction of ADA's potential costs. Aside from education and outreach functions, ADA is essentially a regulatory mechanism; like most regulatory legislation, the major costs are external to its administration.

In sum, it is not currently possible to obtain an accurate estimate of ADA's administrative and enforcement costs, although there are signs that its administration may be underfunded at this time.

Challenge #2: Definitional, Measurement, and Scope-of-Research Issues

a) Definitional issues

Various federal statutes define disability differently, to accomplish varying purposes. There is not a clear operational definition of "disability" based on the ADA, which is problematic from the point of view of enforcement, and also for measuring and tracking progress (National Council on Disability, 2004)

Furthermore, ADA's imprecise definitions of "reasonable accommodation" and "undue hardship" raise difficulties. Some have pointed out that Congress incorrectly assumed that case law under the Rehabilitation Act was uniform and coherent enough to serve as the basis for rulings under the ADA. But, financial "undue hardship" had not been adequately addressed under the Rehabilitation Act which does not cover employers who do not receive government funds, whereas ADA does.

For some classes of persons, the differential nature of "reasonable accommodations" can be especially problematic. The National Council on Disability (NCD), for instance, has highlighted the fact that many accommodations at work and school that might be useful and reasonable for persons with cognitive impairments, are rather abstract and harder to quantify than are most accommodations for persons with physical disabilities. Thus accommodations for persons with intellectual disabilities have yet to be quantified.

Difficulty in collecting valid and reliable data about Americans with disabilities arises from several Supreme Court decisions over the past three years that could dramatically narrow the legal standards for who is a "person with a disability". These decisions complicate assessments of whom the law covers. For research purposes, these decisions will likely play havoc with sample designs, questionnaires and other research techniques.

b) Measurement issues

Perhaps the greatest recognition of research-related challenges in ADA pertains to its employment objectives. President Clinton issued Executive Order 13078 on March 13, 1998, establishing the Presidential Task Force on Employment of Adults with Disabilities to "design and [implement] a statistically reliable and accurate method to measure the employment rate of adults with disabilities as soon as possible..." The Task Force and about 15 federal agencies including Bureau of Labor Statistics, Census, Department of Health and Human Services, and NCD, established the Employment Rate Measurement Methodology Work Group (ERMM). Accomplishing the charge was complicated by having (a) no universally agreed-upon definition of disability, and (b) no data source containing labor force and disability questions that had been tested to determine if people understood them (validity) and if responded consistently over time (reliability). The ERMM's results are to be issued in 2004.

NCD has also pointed to the knowledge gap due to varying approaches to measuring disability, and to inappropriate, inadequate and infrequent surveys, (National Council on Disability,1998) emphasizing that lack of progress in data collection continues to hamper policy analysis.

c) Scope-of-research issues

The ADA is comprehensive, covering: (1) the public and private sectors; (2) various levels of government; (3) much of our nation's infrastructure from the physical or built environment to the communications environment; and (4) millions of individual Americans with disabilities (one estimate is 54 million). Any effort to determine whether the ADA is "working" must be robust yet flexible, using a range of research methodologies to take all major elements into account. No ADA research endeavor has even approximated such an evaluation scheme or framework.

An impact evaluation in general, is a means of judging performance by understanding changes (intended or not) experienced by primary stakeholders (e.g. individuals with disabilities) as a result of social intervention(s) (e.g., anti-discrimination legislation). It can help determine whether the implementation and enforcement of the ADA are achieving Congressional objectives and expectations; whether the objectives and expectations remain relevant over time; and whether the best action strategies have been pursued.

Why should an impact evaluation of the ADA be conducted?

To inform decisions on whether to expand, modify, or eliminate unsuccessful ADA policy or program activities, and to prioritize public actions that increase the fidelity of ADA implementation. In addition, an impact evaluation can contribute to improved effectiveness of ADA policies and programs by addressing the following questions:

• Does the program or policy achieve Congress' intended goals?
• Can changes in outcomes be explained by the program or policy, or are they the result of other factors occurring simultaneously?
• Do program/policy outcomes vary across different groups of intended beneficiaries (males, females, people from diverse cultural backgrounds), regions, and over time?
• Are there unintended effects of the program or policy, positive or negative?
• How effective is the program or policy compared with alternative interventions?
• Is the program or policy worth the resources it costs?

When should an impact evaluation of the ADA be conducted? Immediately. Indeed, the time is long overdue. An impact evaluation demands a significant amount of information, time and resources over a substantial period. It is important to select carefully which ADA policies and programs will be evaluated considering the potential of evaluation results for providing insights for "midcourse" correction, as necessary.

Five questions could help guide the decision of how to focus an ADA impact evaluation:

1. In what way(s) is the ADA strategically relevant for civil rights protections? Policies and programs expected to have the greatest civil rights outcomes for people with disabilities should be evaluated to ensure that public (and private) efforts are on the right track, and allow for any necessary corrections.
   
   
2. Evaluation of which particular ADA policies or programs will contribute most to filling knowledge gaps about what does and does not work in civil rights implementation and enforcement? Impact evaluation can help identify and highlight data-based, validated, and/or cost-effective approaches to providing civil rights protections and decreasing discrimination. Therefore, the decision of what to evaluate could also be guided by how much is known about the effectiveness of a variety of public and private ADA activities.
   
   
3. What levels of analysis are necessary (i.e., individual, regional, nationwide) related to each of ADA's five titles? What approaches and methodologies offer realistic promise that an impact evaluation can be completed?
   
   
4. Are communities testing innovative approaches to ADA implementation and enforcement? Impact evaluation should foster learning. Efforts to improve ADA will require new ideas. Impact evaluations can help to verify pioneering approaches and to decide whether they should be pursued on a larger scale. Policymakers are also interested in whether the ADA has resulted in improved lifestyle outcomes for citizens with disabilities (e.g., employment, homeownership). Therefore, the innovative character of communities' policies and programs should be a strong focal point for evaluation.  One important caveat, however, is that a fruitful impact evaluation will require the selection of sufficiently mature ADA programs and/or policies, across a range of areas: employment, program access, physical accessibility, electronic and information technology accessibility, etc.
   
   
5. Are ADA policies and programs targeted at difficult-to-reach groups, or are the policies and programs expected to have differentiated outcomes (e.g., geographic)? ADA policies and programs targeted to difficult-to-reach groups may encounter social, cultural, economic, and organizational factors that contribute to their success or failure. For these reasons, a well-designed and executed impact evaluation will be particularly important.

How should we proceed? Evaluating ADA's impact requires the best thinking this nation can muster. It would be most prudent for the federal government to identify and convene a core working group of about 15 "experts" representing a range of disciplines. Those experts should have personal and/or professional experience with the ADA and related programs and policies. The group would be responsible for designing the potential contours (methodological approach) of an ADA impact evaluation, to be funded by a federal agency.

On February 24, 2004, NCD announced its interest in evaluating the impact of the Americans with Disabilities Act, and the effects of U.S. Supreme Court cases interpreting that law. NCD specifically expressed its interest in hiring a researcher-contractor to gather input from ADA stakeholders about ADA's impact; gather testimony and documentation regarding the impact of the Supreme Court's decisions on people with disabilities; assemble and organize all testimony into a consistent format by issue area; compile them by state and congressional district; and summarize the findings.

1. Use ADA as the basis for the definition of disability in federal surveys.

The ADA requires viewing disability as dynamic rather than static--as an interaction between an individual with an impairment and the environment rather than as a deficit of an individual. Definitions and measures of disability need to reflect this orientation. In addition, questions about disability issues should be integrated into questions asked of all respondents. For example, disability-related support needs could be included in a list of support needs in a question asked of all respondents, not just those who have identified themselves as having an impairment.

2. Operationalize the nation's goals for people with disabilities, as articulated in ADA, so that data can be collected about the extent to which society is moving toward reaching those goals.

The ADA states that the nation's proper goals for people with disabilities are (1) equality of opportunity, (2) full participation, (3) independent living, and (4) economic self-sufficiency. Statistical measures should be developed for each of these goals. Data should be regularly collected to determine whether the nation is moving toward these goals. The government should dedicate resources to this effort.

3. Ensure that people with disabilities are integrally included in planning, developing, and carrying out disability-related ADA data collection activities.

People with disabilities should be a part of designing and refining data collection instruments, determining questions that will guide analysis, and developing dissemination strategies. Too often, data are generated that are irrelevant or unusable by people with disabilities. People with disabilities are often unaware of what is available. Inaccessible formats prevent meaningful use. People with disabilities should be provided financial support for their participation in the provision of data (e.g., responding to surveys and requests for information), the development of data activities, and the utilization of data. Information should be available in accessible formats in all phases of data collection activities.

4. Evaluate state data systems for elements common to both federal and state data banks, and identify minor modifications that could establish additional common data elements, including key demographic, disability, service and cost variables.

Many informational needs (such as characteristics of low-prevalence populations and outcome efforts) cannot be addressed via federal survey efforts. Significant data on needs, programs, outcomes, and efficacy are found at state and local levels more often than at the federal level. The Federal Government should take the initiative to identify, synthesize, and make available the significant amount of data that exist at state and local levels.

This policy brief overviews the status of knowledge about the impact of the ADA. A thorough analysis of ADA's impact is hampered by the lack of systematic data gathering and impact evaluation studies in the U.S. In fact, policy analysts have had to rely on discrete studies, opinion polls, various statistical sources and their own research interviews to "guesstimate" the role that the ADA has played in one lifestyle area (i.e., employment) only.

Policymakers should ensure that the federal government undertakes a systematic analysis of the gaps in the existing data and research systems as they relate to the ADA, including an identification of data and research strategies that are and are not working effectively, with a view to how the present system of data collection and research could be improved. Such additional data, information, and impact analysis could serve as the basis for capitalizing on the existing strengths of the ADA, as well as addressing those areas of the Act that could be refined. A federally-sponsored impact evaluation encompassing all Titles of the ADA is clearly warranted.

National Council on Disability (January 2004). Improving Federal Disability Statistics. Author: Washington, D.C.

National Council on Disability (April, 1998). Reorienting Disability Research. Washington, D.C. U.S. Government Printing Office.

When it was enacted, the ADA set forth one requirement regarding evaluation or assessment of one aspect of the law. Section 305 of S.933 (the ADA) required the Office of Technology Assessment to undertake a study to determine: 1) the access needs of individuals with disabilities for over-the-road buses and over-the-road bus service; and 2) the most cost-effective methods for providing that access, particularly for individuals who use wheelchairs, through all forms of boarding options.

The study was to include, at a minimum, an analysis of: (i) anticipated demand by individuals with disabilities for accessible over-the-road buses and over-the-road bus service; (ii) the degree to which such buses and service, including any service required under sections 304(b)(4) and 306(a)(2), are readily accessible to and usable by individuals with disabilities; (iii) effectiveness of various methods of providing accessibility to such buses and service ; (iv) cost of providing such accessibility, including consideration of recent technological and cost-saving developments in equipment and devices; (v) possible design changes in over-the-road buses that could enhance accessibility, including accessible restrooms that do not result in loss of seating capacity; and (vi) the impact of accessibility requirements on continuation of over-the-road bus service, with particular consideration of the impact of such requirements on such service to rural communities. Unfortunately, this mandated evaluation was not finally approved.

There are sources of periodic government data about the ADA as a compliance and enforcement mechanism, including:

(a) U.S. Equal Employment Opportunity Commission (EEOC) Complaint/ Findings (1992-present) located at http://www.eeoc.gov/stats/ada-charges.html ,

(b) U.S. Department of Justice (DOJ) (1994-present), located at http://www.usdoj.gov/crt/ada/enforce.htm#anchor201570 , for example; and

(c) U.S. Congress Office of Compliance (COC) Report of Compliance (1998, 2002), located at http://www.compliance.gov/reports-studies/ada_12-02/ada_report.pdf , for example.

Other occasional sources of governmental data and information about the ADA include reports involving:
(a) costs of accommodations (GAO, 1990);
(b) psychiatric disabilities and employment (OTA, 1994);
(c) wilderness accessibility (NCD, 1992);
(d) accessibility and barriers (GAO, 1993);
(e) effects of the law on access to goods and services (GAO, June 1994);
(f) challenges faced by transit agencies in complying with the Act (GAO, March 1994);
(g) consumer opinions about the law's implementation (NCD, 1993, 1995);
(h) people with cognitive impairments and Title I of the ADA (NCD, 1996);
(i) measurement challenges and regulatory concerns raised by selected companies (GAO, 1996);
(j) status of implementation of the law (USCCR, 2000); and
(k) status of enforcement of the law (NCD, 2000).

It should be emphasized that subsequent to the ADA's passage, some members of Congress desired a long-term evaluation of the Act. As noted above, a number of GAO studies were completed. The key goal of this effort was to evaluate whether the main objectives of the ADA were being achieved. This included GAO's investigating, for example, whether access to goods and services under Title III has increased and discrimination decreased since the ADA was passed, and whether public attitudes towards people with disabilities have improved. However, this long-term focus on the ADA by GAO appears to have run its course with the issuance of its last report in 1997. None of the GAO studies has included a focus on the impact of the ADA on its protected class of citizens, namely, Americans with disabilities.

Non-governmental sources of data and research regarding the ADA have been produced by researchers in academia (e.g., DeLeire, 2000; Schwochau & Blanck, 2000), and in other non-profit settings (e.g., National Bureau of Economic Research, 2000; West, 1994).

In most instances, non-governmental data and research have addressed the issue of employment or unemployment and the 'negative' impact of the ADA on people with disabilities. Attempts at assessing the employment or unemployment effects of the ADA, however, have been fraught with fundamental definitional and methodological problems (See, Hale, 2001; Kirchner, 1996, 1999 ; Levine, 2000; Stapleton & Burkhauser, 2003; Tolin & Patwell, 2003, for good analyses of the issues). Other non-governmental data and research have addressed the issue of ADA awareness and/or satisfaction by people with disabilities and have not attempted to address the impact, per se, of the civil rights law on peoples' lives.

In essence, non-governmental data and research, to date, has been: (a) small scale and non-representative of the population of Americans with disabilities as a whole, (b) non-comprehensive in its breadth and scope of assessment, and (c) inconclusive in its findings and analyses.

Bound, J. and Waidman, T. (2000). Accounting for recent declines in employment rates among the working-aged disabled. Workng Paper 7975. Cambridge, MA: National Bureau of Economic Research.

DeLeire, T. (2000).The unintended consequences of the Americans with Disabilities Act. Regulation, 23(1). Washington, D.C.: The Cato Institute.

GAO - General Accounting Office (January, 1990). Persons with Disabilities: Reports on Costs of Accommodations. Briefing Report to Congressional Requesters(GAO/HRD-90-44BR) Author: Washington, D.C.

GAO - General Accounting Office (May, 1993). Americans with Disabilities Act: Initial Accessibility Good But Important Barriers Remain. Report to the Chairman, Subcommittee on Select Education and Civil Rights, Committee on Education and Labor, U.S. House of Representatives. (GAO/PEMD-93-16) Washington, D.C.: U.S. Government Printing Office

GAO - U.S. General Accounting Office (March, 1994). Americans with Disabilities Act: Challenges faced by Transit Agencies in Complying With the Act's Requirements. Report to Congressional Requesters. (GAO/RCED-94-58) Washington, D.C.: U.S. Government Printing Office

GAO - General Accounting Office ((June, 1994). Americans with Disabilities Act: Effect of the Law on Access to Goods and Services. Report to the Chairman, Subcommittee on Select Education and Civil Rights, Committee on Education and Labor, U.S. House of Representatives) (GAO/PEMD-94-14) Washington, D.C.: U.S. Government Printing Office.

GAO - General Accounting Office ((November, 1996). Regulatory Burden: Measurement Challenges and Concerns Raised by Selected Companies. Report to Congressional Requesters. (GAO/GGD-97-2) Washington, D.C.: U.S. Government Printing Office.

Hale, T. (2001). The lack of a disability measure in today's Current Population Survey. Monthly Labor Review, June, 38-40.

Kirchner, C. (1996). Looking under the street lamp: Inappropriate uses of measures just because they are there. Journal of Disability Policy Studies, 7 (1), 78-90.

Kirchner, C. (1999) "Evidence and Rationales in the Debate over the ADA's Effect on the Status of People with Disabilities," paper presented August 8, at the 1999 Annual Meeting of the American Sociological Association, Chicago, Illinois.

Levine, L. (2000). The Employment of People with Disabilities in the 1990s. Washington, D.C.: Congressional Research Service, The Library of Congress.

NCD - National Council on Disability (December, 1992). Wilderness Accessibility for People with Disabilities: A Report to the President and the Congress of the United States on Section 507(a) of the Americans with Disabilities Act. Washington, D.C.: U.S. Government Printing Office.

NCD - National Council on Disability (April, 1993). ADA Watch-Year One. Washington, D.C.: U.S. Government Printing Office

NCD - National Council on Disability ((January, 1995). The Americans with Disabilities Act: Ensuring Equal Access to the American Dream. Washington, D.C.: U.S. Government Printing Office

NCD - National Council on Disability (January, 1996). Cognitive Impairments and the Application of Title I of the Americans with Disabilities Act. Washington, D.C.: U.S. Government Printing Office.

NCD - National Council on Disability (June, 2000). Promises to Keep: A Decade of Federal Enforcement of the Americans with Disabilities Act. Washington, D.C.: U.S. Government Printing Office

0TA - Office of Technology Assessment U.S. Congress (1994). Psychiatric Disabilities, Employment, and the Americans with Disabilities Act. (OTA-BP-BBS-124). Washington, D.C.: U.S. Government Printing Office.

S.933 - Senate Bill 933, Section 305..The Americans with Disabilities Act

Schwochau, S. and Blanck, P.D. (2003). Does the ADA Disable the Disabled? Industrial Relations, 42, 67-77.

Stapleton, D. and Burkhauser, R. Eds. (2003) The Decline in Employment of People with Disabilities: A Policy Puzzle, Kalamazoo, MI: W.E. Upjohn Institute for Employment Research.

USCCR - U.S. Commission on Civil Rights (October, 2000). Sharing the Dream: Is the ADA Accommodating All? A report on the Americans with Disabilities Act. Washington, D.C.: U.S. Government Printing Office.

West, J. (1994) Federal implementation of the Americans with Disabilities Act: 1991 -94. New York, NY: Milbank Memorial Fund