Disability Studies Quarterly
Fall 2005, Volume 25, No. 4
<www.dsq-sds.org>
Copyright 2005 by the Society
for Disability Studies


Ten States' Web Services and Policies on Web Accessibility for People with Disabilities

David W. Klein, Ph.D.,
Director of Technology
Law, Health Policy & Disability Center
University of Iowa
E-mail: David-klein@uiowa.edu

Daniel Kresowik
Law, Health Policy & Disability Center
University of Iowa
E-mail: Daniel-kresowik@uiowa.edu

LeeAnn McCoy
Law, Health Policy & Disability Center
University of Iowa
E-mail: leeann-m-mccoy@uiowa.edu

The program of research described herein is supported by a grant to LHPDC from the Information Technology Technical Assistance and Training Center. The views herein reflected are those of the authors only and not of any funding agency or any other entity. Many colleagues provided valuable suggestions on this article including James Schmeling. For additional information on the resources and policy briefs referenced by the LHPDC, or to direct correspondence, see http://disability.law.uiowa.edu.

Abstract

Six Web-based services from 10 state websites were assessed for accessibility for people with disabilities. Five states with strong web accessibility policies and five with relatively weak accessibility policies were selected for comparison. Among a variety of measures for accessibility, including "Bobby," strong policy state websites were compared against weak policy state websites. Strong policy states showed a difference in the number of links on pages, their use of JavaScript, and a trend in how they passed Bobby priority 1. However, no other evidence of any apparent effect from strength of policy was found for accessibility on these websites. Overall, states showed similar levels of accessibility to other public websites in other similar studies. Issues and ways to improve state website accessibility are explored.

Ten States' Web Services and their Policies on Web Accessibility for People with Disabilities

Introduction

State governments are providing services via the World Wide Web such as job listings, tax forms, and voter registration. These Web-based services reach more people and may be updated more quickly than paper versions, at a reduced cost. But, as brick-and-mortar locations present barriers to physical access, the Web presents its own barriers to individuals with disabilities. Under Section 508 of the Rehabilitation Act of 1973 as amended in 1998, Congress mandated that all new federal procurement would be accessible to people with disabilities as of June 2001, including web pages. Subsequently, many states have passed their own versions of Section 508.

All states have created guidance on how state web pages should be made accessible to people with disabilities (see ITTATC, 2002; Golden & Buck, 2003). However, West (2004a) determined that only a third (37%) of state websites satisfy W3C standards of Web accessibility. Similarly, Rowland (2000) reported that roughly one quarter of postsecondary institutions' home pages were accessible. In a series of studies by Schmetzke (2001), postsecondary education sites ranged from 15% to 23% passing Bobby priority 1.

Fagan & Fagan (2004) note that about two thirds of visitors to government websites seek information on state public policy issues. About one fifth (21%) received information that influenced their vote in an election. Almost three fourths (71%) reported their use of a state website "improved the way they interact with government at least a little" (p. 66).

Coyne & Nielsen (2001) recommend that web developers pay particular attention to how their sites affect people with disabilities. People without disabilities completed common Web-based tasks 3 to 6 times more often than people with disabilities who were using assistive technology. These tasks included finding facts, buying an item, getting travel information, and comparing products. Individuals with disabilities averaged more than twice the time to perform tasks, they had 3 to 6 times more errors, and they experienced more frustration, less confidence and less satisfaction than the control group.

The current study explored the relationship between the strength of state accessibility policies and the accessibility of their websites. Six Web-based services were studied: submitting personal state income taxes online, accessing information about employment, obtaining a license for a small business, obtaining income tax forms, paying property taxes online, and obtaining voter registration information. The accessibility of these six services was compared for five states with strong accessibility policies and five states with relatively weak policies.

Web Accessibility and the Law

Title II of the Americans with Disabilities Act (ADA) requires that "no qualified individual with a disability shall, by reason of such disability, be excluded from participation in or be denied the benefits of the services, programs, or activities of a public entity" (ADA, 1990). Passed in 1990, before the emergence of the Internet, the ADA does not refer to Internet access. The issue of whether private websites are subject to the antidiscrimination provisions of Title III of the ADA (the law's public accommodation provisions) has been the subject of litigation (Blanck & Sandler, 2000; National Federation of the Blind v. America Online, Inc., 1999; Access Now, Inc., v. Southwest Airlines, Co., 2002). However, the issue of whether state websites are subject to the ADA's Title II provisions has not been addressed.

Research on Web Accessibility

The two dominant standards for Web accessibility are the Web Content Accessibility Guidelines (WCAG) and federal regulations developed by the Access Board for Section 508. WCAG, developed by the Worldwide Web Consortium's (W3C) Web Accessibility Initiative (WAI), specifies general ways for using Hypertext Markup Language (HTML) to provide the best experience to the widest possible audience. The WCAG is divided into three priority levels for accessibility errors on web pages.

Priority 1 errors are showstoppers – barriers that prevent some people from access to information altogether. For example, a graphic object on a page without a text alternative, text-based label, or audio description prevents individuals who cannot see the object from knowing the object's content and purpose.

Priority 2 errors produce barriers for access to information but do not prevent access entirely. For example, a user who cannot see the computer screen may not realize that a new browser window has opened after clicking a link. When the user wants to return to the original page, the browser's Back button will not work. Users may quit their browser and start over. In this case, the information in the original and new page is otherwise accessible, but a design characteristic of the page discourages its use.

Priority 3 errors allow users with disabilities to access information without significant barriers but do not provide parity with other users in terms of efficiency and effectiveness. For example, a page that contains hundreds of links may seem unorganized to some individuals and will be difficult to navigate if users need the tab key to navigate the links.1 In contrast to the WCAG, the Access Board regulations are less specific and comprehensive, although they frequently mirror the WCAG specifications and reference WCAG's language and model for exemplary purposes.

Some of these guidelines may be tested on web pages using software applications called accessibility checkers. For example, an accessibility checker can look for alternative text, which conveys the meaning of a graphic to people with visual disabilities. In the absence of a text alternative, the accessibility checker software can alert the developer that the web page is not accessible. However, the checker program cannot determine whether the text alternative sufficiently describes the image. This kind of software application is able to determine a condition on a web page that is not accessible (no alternative text for an image), but it is unable to determine whether the condition is accessible with any certainty without human assistance.

The most commonly used accessibility checker is "Bobby," based on WAI guidelines. Studies using Bobby have focused on high school websites (Klein et al., 2003), post-secondary institution websites (Schmetzke, 2004; Thompson, Burgstahler, & Comden, 2003; Schmetzke, 2001), federal and state government sites (West, 2004a; Golden & Buck, 2003), large businesses (Zeng & Parmanto, 2004; Sullivan & Matson, 2000), and international government sites (West, 2004b). Bobby statistics from such studies provide a means of comparing research.

Alternative Text for Graphical Objects

For many individuals with disabilities, a formidable barrier is the graphical user interface (GUI). The GUI is the underlying structure for most computer operating systems such as Microsoft Windows. Computer users interact with the GUI by using a mouse to click on objects they see on a computer screen. The GUI creates access problems for people with visual limitations who cannot see the objects on the screen and individuals with limited mobility who cannot easily manipulate the mouse (National Council on Disability, 1996).

Many users with visual disabilities use computer programs called screen readers. These programs convert text to audio, where they read aloud the contents of windows, names of buttons and other controls, and typing on the keyboard. Graphic images on the screen must include text alternatives for the screen reader to read.

Nearly all computer programs use the graphical user interface. Because the Web was designed primarily as a graphical medium, web pages are designed to present information that includes pictures and tables. Web designers must provide text alternatives to convey nontextual content information to make the page accessible. WAI guidelines specify ways to convey this content information through HTML. For example, text alternatives can be placed in "ALT tags," hidden areas of a web page that contain alternative text.

Because graphical content without text alternatives is a complete barrier for individuals with visual disabilities, standards bodies and accessibility researchers have focused on alternative text as crucial to accessibility on the Web. Adding alternative text also can be a relatively efficient first step to dramatic improvement in accessibility. In a study of high school home pages, Klein, et al., (2003) found that only 7.6% (12) of home pages passed Bobby priority 1, out of 157 home pages studied. If all these pages included ALT tags for each graphic, a relatively simple and cost effective process, 91.1% (143) would have passed Bobby priority 1.

Keyboard Access

Using browsers, people view web pages with the help of a mouse, clicking on the scroll bar to scroll down the page and clicking on hyperlinks, images, or text to display other pages. This latter process is called "navigation." Most individuals who cannot use the computer mouse will use the keyboard to interact with web pages. This group includes those with visual disabilities or learning disabilities, such as blindness, low vision, and dyslexia, as well as individuals who have trouble manipulating a mouse because of arthritic conditions, repetitive stress injuries, and conditions affecting fine and large motor control.

All dominant browsers support the use of the keyboard as an alternative to the mouse using the tab and enter (or return) keys on the keyboard to highlight and navigate hyperlinks. The tab key is used to advance through hyperlinks on a page and the enter key to "click" on the hyperlink. People using screen readers and other assistive technologies use the keyboard for a variety of functions that make web browsing more efficient.

A range of barriers exists for people who use the keyboard to access the Web. Some pages are designed so that links are difficult to use via the keyboard. The tab key may take users through the links on a page seemingly haphazardly, causing confusion. Sometimes pages are designed so that merely selecting a link by tabbing to it causes the browser to navigate to the link's destination, effectively preventing a keyboard user from tabbing down a page past such a link. JavaScript or other types of features embedded in web pages can prevent users from tabbing to links. In some web pages, the number of links is so great that tabbing becomes burdensome to the point where reaching the desired content on a page may be prohibitive, especially when this kind of tabbing pattern is repeated throughout a website. The New York Times home page (www.nytimes.com) may require 80 or 90 tab keystrokes to get through the sidebar (which includes links to subsections of the website as well as advertising) to arrive at the first line of content. Subsections of the nytimes.com website use roughly the same sidebar and require similar amounts of tabbing to get to content. This kind of page design is time consuming for people who rely on the tab key to navigate web pages and causes unwanted repetitive motion. In one study, screen reader users expressed dismay over browsing pages with hundreds of links (Theofanos & Redish, 2003).

Other Barriers

For people with disabilities who browse the Web, excessively long pages create their own barriers. Besides excessive tabbing, large numbers of graphics present barriers. Where a few graphics can enhance usability, excessive graphic images can take a long time to download, increase page complexity, take a long time for screen readers to read, and make pages difficult to understand. Large numbers of graphics used for links aggravate such barriers. Further, images often called "spacers" are sometimes used for page layout and invisibly occupy the "white space" of the page, irrelevant to the content. Spacers add complexity to web pages if screen readers identify them.

Finally, JavaScript and other types of scripting, which add functionality to web pages, can have a negative effect on accessibility. First, some browsers, such as Lynx, a text-based browser used frequently by people with disabilities, do not support JavaScript. Second, JavaScript does not currently have standards or guidelines for accessible design the same way that HTML has WAI guidelines (WCAG).2 Although JavaScript adds significant functionality to a web page, which can be alluring for developers, little support exists for helping developers design JavaScript to create accessible pages. Third, accessibility checkers do not check JavaScript for accessibility, so it is possible for web pages to pass Bobby and other accessibility checkers but be inaccessible for all practical purposes.

Method of Study

Materials

Sources for web pages were services provided by the websites of ten states. Five of the states represented states with strong Web accessibility policies and five represented states with relatively weak accessibility policies. The six Web-based service activities were

1. Electronically submitting personal state income taxes;
2. Accessing information about employment services and job listings;
3. Obtaining a license for a small business;
4. Obtaining income tax forms (1040 EZ or the equivalent);
5. Electronically paying property taxes; and,
6. Obtaining voter registration information.

The states' home pages were the starting points. The number of clicks through the links until the target page was reached was one measure of accessibility. If a service required that users enter personal information to continue, such as a social security number or ZIP code, the assessment ended at that page. This resulted in a list of web pages for each service for each state, which the states' constituents would have to view, interpret, and navigate so that they could use these services, totaling 260 pages. Of these 260 pages, 147 unique pages were evaluated after eliminating redundant pages and text documents (Acrobat or Word files),

Strong versus Weak State Accessibility Policies

State websites were chosen based on the Information Technology Technical Assistance and Training Center database of state accessibility policies (2004).3 Factors used in determining whether a state's accessibility policy was strong or weak were 1) specificity, 2) use of a widely accepted standard, 3) scope, 4) level of perceived flexibility, 5) inclusion of explicit deadlines, and 6) consequences for noncompliance.

Specificity. A strong policy should be more specific than general. Specific policies included sections with detailed descriptions of possible accessibility issues such as color contrast, graphics, moving content, page layout, and text-only pages, and possible solutions for making the pages more accessible. General policies were briefer, listing the issues with no descriptions or solutions, or stating that the state "supports" accessibility, or that state websites should be "designed for" compliance. Arizona, Colorado, and Illinois had specific policies, so these states were rated as strong in this category. The other states' policies either lacked specificity altogether or included less than two printed pages of specifics, as opposed to over twenty pages of specifics for the strong policies.

Use of standards. A strong policy should reference existing standards, but formulating a strong policy might involve customizing those guidelines to suit the needs of the individual state. The policies of Arizona, Arkansas, Colorado, Illinois, Kentucky, Iowa, Mississippi, and Oregon were considered strong in their use of standards because they were explicitly based on WCAG or Section 508. The other states' policies were considered weak in this category because they were either unclear about the basis for their policies or referred to a standard in a general way.

Scope. State policies were evaluated for scope, covering a broad range of entities. A policy's effect would be restricted if it applied only to a single state agency. The policies of Arizona, Arkansas, Colorado, Illinois, and Kentucky were rated strong in this category because they applied to more than just the state agencies, such as state commissions, education systems, and individuals responsible for information technology implementation. Other states' policies were categorized as weak for scope.

Requirements. An inquiry was made into whether the policy contained explicit requirements for the covered entities or mere guidelines. The policies of Arizona, Arkansas, and Kentucky specified that their stated accessibility standards were required, and thus, these policies were considered strong in their requirements. Iowa's policy was ambiguous in that it required compliance to ("must meet") what were termed "[r]ecommended guidelines," which caused it to be downgraded to a weak policy.

Deadlines. Colorado, Kentucky, and Oregon were considered to have strong policies in this category because they had explicit dates when the policy goes into effect or deadlines for compliance. Compliance deadlines either do not exist or could not be found for Arizona, Arkansas, Illinois, Iowa, Mississippi, New Mexico, and Ohio.

Consequences for noncompliance. The final factor for strong policies included a complaint process, whether the policy included sanctions for noncompliance, and whether there was a specific person to contact or a mechanism (e-mail, phone, fax, office address) listed for questions or complaints. The policies of Arkansas and Kentucky were considered strong in this category because they included a complaint process, listed specific consequences for noncompliance (injunctive relief), and provided a contact e-mail address for complaints or feedback. Weak policies omitted one or all of these factors.

Arkansas, Arizona, Colorado, Illinois and Kentucky were selected as strong policy states because their Web accessibility policies demonstrated most of the characteristics of strong accessibility policies. Iowa, Mississippi, New Mexico, Ohio and Oregon were selected as weak policy states because their Web accessibility policies lacked or partially implemented most of the characteristics above as compared to the stronger policies.

Selecting pages

To determine which pages to include in our dataset, a researcher was instructed to start at each state's home page and look for each of the six services, review each page, and click on the most likely link to the page that contained the target, keeping track of each page address and link that was clicked. Although the researcher occasionally took some erroneous paths and had to backtrack, these extraneous pages were not included in our dataset. The pages represent the path that a reasonable person would take to reach each of the state's six services.

Coding Protocols

The state web pages were coded for specific accessibility features or barriers. Two research assistants at a major state university coded the pages. Coders were trained on the barriers to web access, and how to use the research group's tools for compiling accessibility-related data, such as ALT tags, tables, or clickable hyperlinks. An online software application, developed by the team, counted ALT tags, hyperlinks, tables, graphics, and third-party applications, increasing the speed and reliability of the coding.

The coding protocols were developed using an interactive, iterative process whereby coders reviewed and resolved coding issues on 10 pages spanning different services. During this process, coders refined ways to count discrete elements of web pages, such as number of hyperlinks, ALT tags, and images. In addition, protocols and the internal application were refined to increase rater reliability.

The coding of the web pages was organized into the following five areas:

  1. descriptive data, such as the title of each page, description of the hyperlink used to navigate to the next page in the sequence, and the type of link.
  2. feedback from the Bobby accessibility checker. The desktop version of Bobby was applied to each page, providing a pass/fail assessment on each WCAG priority for the page, as well as detailed feedback on specific WCAG guidelines.
  3. counts of ALT tags, hyperlinks, tables, third-party applications, images, use of JavaScript, frames, form elements, and cascading style sheets.
  4. a comparison of ALT tags on each page, as well as a way of counting graphics used for navigation paired with ALT tags.
  5. accessibility of Adobe Acrobat (PDF) files.

The internal application parsed the ALT tags and displayed them in an easy-to-read format so that researchers could evaluate the contents of the tags, such as whether the tags were merely blank, used file names, asterisks (usually indicating bullet points), or appropriate descriptions of graphics. In addition, researchers could compare ALT tags to see if they contained redundant information. The internal application displayed hyperlink HTML so that researchers could determine how many links used graphics and, further, whether the graphics had an associated ALT tag.

Basic accessibility for PDF files

Three important factors that indicate whether a PDF file could be accessible were examined: 1) the version number of Adobe Acrobat that produced the file, 2) whether the "Enable Accessibility" feature had been selected, and 3) whether the file had been enabled for tags.

Results

Of the 168 unique pages, the number of pages for strong accessibility policy states (85) roughly equaled those for weak accessibility policy states (83). Twenty of the pages were PDF files and one was a Word document. These pages were not included for most of the analyses because they do not use HTML.

On average, people would have to view 4.3 pages to reach the service they were looking for. Arkansas and Kentucky required access to the fewest number of pages (3.5), and Iowa required the most (5.8) (see Table 1). There is no substantial difference between the average number of pages viewed for each service for strong (4.1) compared to weak (4.6) state accessibility policies (p=.105).

Table 1. Number of pages, including the home page, viewed to reach each web-based state service.

State

Electronic Filing of Income Taxes

Employment Services and Job Listings

Obtaining Business License

Obtaining Income Tax Forms

Voter Registration Information

Paying Property Taxes

Mean

AR

3

4

3

6

2

3

3.50

AZ

4

6

5

4

4

4

4.50

CO

7

5

5

4

3

4

4.67

IA

7

4

5

7

6

6

5.83

IL

5

5

4

3

3

5

4.17

KY

4

2

3

5

4

3

3.50

MS

6

2

3

6

4

5

4.33

NM

4

6

5

5

4

3

4.50

OH

6

4

3

5

5

5

4.67

OR

5

3

2

4

4

4

3.67

Mean

5.10

4.10

3.80

4.90

3.90

4.20

4.33

Bobby Scores

Of the 147 unique Web pages, 2 passed all three Bobby priorities. Both of these pages were on the Kentucky state website. No page passed both priorities 1 and 2. Three pages passed both priorities 1 and 3. Of these, again two were on the Kentucky website; the third was on the Iowa site.

Priority 1

Of the 147 unique pages evaluated with Bobby, 40.8% (60) passed Bobby priority 1. There was no substantial difference between strong and weak states on the number of web pages that passed Bobby priority 1 (p=.087). The failure of testing the existence of alternative text for all images on the page,4 determined the failure of 67.8% of pages on priority 1. If all pages in this group had included appropriate alternative text (ALT tags) for all images, about twice as many, 80.9%, of the pages would have passed priority 1.

Bobby Priority 2

Five pages passed Bobby priority 2, two pages from strong states and three from weak. Unlike priority 1 results, failure on more than one guideline was usually a cause for failure on this priority. Although not the exclusive reason for failing, almost all pages that failed priority 2 (95.1%) failed the guideline that recommends use of relative sizing rather than absolute sizing,5 which allows page layout to respond appropriately to changes in the browser window size, text size, and other changes in a browser environment.

Bobby Priority 3

Five pages passed Bobby priority 3, four from the strong states and one from the weak states. As with priority 2, nearly all pages failed to pass more than one guideline. Of the pages that failed to pass priority 3, three guidelines were missed frequently: identifying the language of the text (80.0%), providing a summary for tables (80.0%), and avoiding separating links with white space (72.5%).6

Other Accessibility Issues

The WCAG recommends that Web developers use a protocol called "cascading style sheets" (CSS) to lay out web pages. This protocol ideally separates the content of a page from its format, allowing the message (the content) to remain stable while users manipulate how the message is delivered through their browser. Thus, in theory, users who want to view web pages in large print can increase the size of their font without appreciably affecting accessibility of the content. There was no significant difference between strong and weak states on use of CSS (50 pages each).

The number of links on the web pages ranged from 0 (2 pages) to 485 (1 page), averaging 64.3 links per page. There was a significant difference between the average number of links for strong (78.8) and weak states (53.3, p=.020). The states with strong accessibility policies tended to have more links per page.

We also looked at the number of links where users click on graphics rather than text. Here, the use of alternative text that adequately describes the graphic is crucial. Of the 147 pages, 80.3% (118) used at least one graphic link. These pages ranged from 1 to 91 graphic links, averaging 9.3. There was no significant difference between strong (10.1) and weak (8.5) in number of graphic links per page (p=.395).

Without proper ALT tags, graphic links can be dead ends for some people for site navigation to important linked information. Of the 147 pages, 21 (14.3%) had at least one graphic link without an ALT tag. There was no significant difference between strong policy states (12) and weak policy states (9) on pages that lacked at least one ALT tag for a graphic used for navigation (p=.780).

In evaluating the number of tabs required for keyboard navigation to the state services, the redundant pages were included, because the amount of tabbing would be different for navigating to each service. Out of these 260 pages, we eliminated 26 pages because they were the target pages for the service where the user would not have a reason for further navigation. Sixteen pages were removed from this analysis because links could not be reached at all by tabbing. This occurred when JavaScript was used and did not allow tabbing past a certain point in the page. For the remaining 218 pages, the number of times we had to press the tab key to reach the target link ranged from 0 (4 pages) to 206 (3 pages) tabs, where fewer tab key presses was more desirable. The mean was 28.2 overall, with a mean of 27.8 for strong policy states and 28.4 for weak policy states, not a significant difference (p=.897).

The number of images embedded into the web pages was analyzed. The total number ranged from 0 (3 pages) to 1,054 (1 page) images, with 32% (47) of the 147 pages containing more than 25 images. There was no significant difference between the average number of images per page for strong (50.3) compared to weak (42.1) policy states (p=.625).

Of the 147 unique HTML pages, 77.6% (114) included JavaScript. Though trending in the predicted direction, strong policy sites were not substantially different (83.3%) from weak policy states (72.0%) in using JavaScript (p=.101).

Acrobat files

A separate analysis of the 20 Acrobat (PDF) files, primarily tax forms, was conducted. Of the 20 files, 20% (4) were created as compatible with version 3 of Acrobat. Fifteen percent (3) were compatible with Acrobat 4, 55% (11) were compatible with version 5, and 10% (2) were compatible with version 6. All PDF files studied were enabled for accessibility. None of the PDF files was enabled for tags.

Discussion

Compared to prior studies of state website accessibility (West, 2004a; Golden & Buck, 2003; Fagan & Fagan, 2004; Rowland, 2000; Schmetzke, 2001), with approximately 40% of the unique Web pages in the sample passing Bobby priority 1, this study shows a somewhat higher pass rate over similar organizations. This higher rate suggests improvement in web accessibility, a positive sign for state web-based services. If developers focused more attention to alternative text — even if only for the individual graphics shown on their web pages — the pass rate for priority 1 would be much higher. In addition, state web developers generally seem to have significant knowledge about web development, as suggested by the extensive use of JavaScript, and these kinds of sophisticated technologies do not seem to hinder accessible page design in a measurable way.

On the other hand, pass rates on Bobby priorities 2 and 3 are low, which suggests that many developers are focusing on providing basic accessibility and not worrying about other levels of accessibility for people with disabilities. Further, many of the web pages we studied were dense, with large numbers of graphics and many links. Not only are these issues not tested by accessibility checkers, at the extreme end they raise concerns beyond the scope of this paper about usability for all Web users.

Only one characteristic of the websites in this study showed a strong difference between the strong and weak policy states, the number of links per page, where strong policy sites tended to have more links. Unfortunately, large numbers of links tend to create barriers to accessibility.

Besides the number of links, we did find two characteristics of the websites that showed trends. First, passing Bobby priority 1 tended to occur more frequently on strong state policy sites. This is encouraging and suggests that stronger policies have some effect on web development. Second, strong policy sites tended to have more pages that used JavaScript. To see whether the use of JavaScript was related to accessibility, a correlation was performed between the use of JavaScript and passing priority 1, which showed a moderate positive relationship (r = .223, p<.01). This indicates that web pages using JavaScript are more likely to pass priority 1.

As a possible explanation for this, developers in strong policy states may be more educated and skilled in web development on average, using more sophisticated technologies and creating more complex pages. The greater use of JavaScript and, arguably, the greater number of graphics and links on strong policy websites could be evidence for this. With better knowledge about web development, these developers may be more aware of accessibility, or at least of Bobby. This awareness of Bobby is especially likely in environments where state policy explicitly refers to WCAG for web accessibility and Bobby as an automated way of checking these guidelines. With Bobby as a checking tool, they may put more effort into passing Bobby, which will probably improve page accessibility, but will not address the barriers that Bobby is not capable of finding. Another possible explanation is that experienced developers may be more likely to use best practices in web development, improving usability in general and improving accessibility as a byproduct.

Large numbers of links

To get a better picture of how accessibility plays out in practice, we examined pages that stood out because of some unusual characteristics. For example, the pages with the highest number of links tended to be ones that showed either a long list of downloadable tax forms or provided links to a myriad of departments or state services. The eighth highest number of links (204) was one state's home page, which seems to be designed for comprehensiveness, including a list of departments and even links to the weather for select cities. This page was labeled as "Bobby approved" as well as "Section 508 Bobby approved" (i. e., passes Bobby according to Section 508 rules), although it passed only priority 1 of Bobby.

Of the other pages evaluated, some pages with large numbers of links included shortcuts at the top of the page that would jump a user to a particular link on the page. One shortcut was an alphabetical index, such as one state's Workforce Center page, with the sixth highest number of links in the study. This page included an extensive list of workforce centers in the state. Clicking on a letter of the alphabet high in the page took a user directly to the first city that starts with that letter, saving many tab keystrokes or saving time listening to the screen reader read the names and locations of workforce centers in cities irrelevant to the user's needs.

Another shortcut was a search box, such as the one on the page with the second highest number of links, a page of tax forms for one state's Department of Revenue. Here, users could type the number of the form they wanted to download and the search engine provided a list of search hits.

Large numbers of graphic images

We were intrigued by one page where we found an unusually large number of graphics (1,054). We reviewed the HTML code for this page and discovered that the page included 35 ALT tags that use descriptive alternative text, such as "Mississippi.gov logo and photo of state capitol" or "Search by Topic" for an image used for navigation. The page included another 41 ALT tags that contain no text. This is a common and appropriate accessibility technique that allows screen readers to skip over noncontent images without reading them. However, the other 978 images had no ALT tags and would not be ignored by a screen reader. For images that do not have an ALT tag, most screen readers read the file name of the image. On this page we found 1,017 images with the name "images/nothing.gif." All of the images that omit ALT tags were these "nothing" images.

Acrobat (PDF) files

The wide variation of Acrobat version compatibility among the PDF files reflects increasingly sophisticated accessibility tools available to web developers, with versions prior to 4 having no accessibility features. In this study, the presence of the accessibility option, which is turned on by default, in all files ensures only that the text is available to screen readers but does not indicate whether the text is understandable. Further, complex documents such as forms are unlikely to be readable unless they are tagged. Thus, the lack of tagging in the PDF files, and the lack of accessibility features in version 3 compatible files, means that these documents are not designed for use by people with disabilities, although the text may be available to screen readers.

Text-only as an option

Oregon's website was problematic for our methodology. A person using a screen reader probably would not be able to tab to most of the content links on the home page. The content was organized by main headings with a small plus symbol (+) to the left of each heading. Tabbing to the plus symbol of a heading and pressing the enter key opened a tree of subtopics, which could then be tabbed through. However, a person using a screen reader would likely have difficulty understanding how the page works.

We also noted a major accessibility feature of the site not found on the other sites in this study. The pages we viewed contained several links to a comparable text-only page. The text-only pages on this site were pages generated with an application called LIFT Text Transcoder, a commercial application that automatically generates a text-only version of a web page in real time. With this application (LIFT), the content remains current, and it can be a good alternative for many people.

Improving accessibility through policy

Creating an accessibility policy is an important step in ensuring accessible web pages for an organization. It brings stakeholders to the table to discuss and learn the issues. Participants in the discussion return their knowledge to their corners of the organization. Further, the existence of a policy is a symbol of an organizational commitment to accessibility. However, the existence of a policy — even a strong one — is not a guarantee that web pages will be accessible, as shown in this study.

The dynamics of Web accessibility

Improving website accessibility at the organizational level can be difficult, in part, because of the complexity of the issues. Rowland summarizes the complexity of attaining accessibility in websites. For design and development, authoring tools that are used to create web pages and web applications need to support the creation of accessible websites. Second, user agents, applications that display web-based information, including browsers as well as cellular telephones, personal digital assistants (PDAs), and assistive technology, such as JAWS or ZoomText, must work together in a complicated, delicate collaboration to present information coherently and accessibly. Third, designers and developers have to be educated in the design and development of accessible web pages and motivated to use that knowledge, and users have to be educated in how to use browsers and assistive technologies to benefit from online information and web-based programs (Rowland, 2000).

Further, Yu (2002) has noted a persistent and incorrect myth that assistive technology will inherently handle any accessibility issues, which provides a reason for avoiding careful design of usable web pages. For web pages that have not been developed for usability and accessibility, assistive technology will not ensure adequate — or any — communication of information.

Other organizational factors

Even if the system of developers, technology, and users were to come together and work smoothly, other organizational factors get in the way of the development of accessible websites by states. Some factors include the strength and clarity of laws and policies that address accessibility, state organizational structure, the climate of web development, and accountability processes.

Whereas federal Section 508 regulations are relatively clear, state accessibility laws and policies may be less clear. In the past, some states passed accessibility laws or developed policies that pertained only to certain agencies (Fagan & Fagan, 2004). In this study, Mississippi accessibility policy suggests that the policy pertains only to pages under mississippi.gov.

As we evaluated accessibility policies, we discovered ambiguities in language that may contribute to barriers. A distinction between "requirements," "guidelines," and "tips" or "examples" may not exist. While policy writers do not want to restrict development on new, robust technologies by too much specificity in their requirements, failure to specify clearly what is required and ways to measure development efforts at satisfying these requirements can lead to problems with compliance.

Another issue is the perceived locus of control over the display of information. IT providers tend to determine how information is designed and displayed, as opposed to allowing the users of the websites to determine what works for them (Fagan & Fagan, 2004). The reason for this may be that with higher levels of expertise may come more tendency to trust the developer's experience rather than user feedback, which can be expensive and difficult to collect. This expertise is not necessarily a drawback, however. Schmetzke (2001) suggested that library websites, which used much more JavaScript than Library Science department sites, probably required more technical sophistication, which may have been a factor in their higher Bobby scores.

Finally, consequences for noncompliance may not be clear or existent. Contact information for the person responsible for web development may not be available to the public. Outsourced work may not require compliance or at least compliance at the same level as in-house work (Fagan & Fagan, 2004). In addition, studies that rank websites do not always include accessibility in their ranking criteria (Fagan & Fagan, 2004). If consequences for noncompliance are not spelled out in an accessibility policy, or if the accessibility policy appears ambiguous about requirements, developers may be less assiduous about accessibility. And, where standards are based on a narrow or incomplete metric, such as Bobby priority 1, developers may be more likely to focus their efforts on passing the metric, as shown in this study, without concern for a more realistic and complete level of accessibility.

Maine's experience

In the spring of 2005, the state of Maine completed a redesign of its state Web portal, www.maine.gov, with accessibility as a major focus. Hokkanen, Record, & Wood (2005) described the experience. The state launched several initiatives that would accomplish several state IT goals, which included statewide web accessibility: 1) A state web accessibility subcommittee was established to develop and determine statewide standards for web development, to establish and maintain an evaluation methodology for technology, to provide periodic updates, to write annual updates, and to work with the state CIO to develop issues related to web standards; 2) the state CIO, the top IT official in the state, was given the mandate of champion for the accessibility initiative; 3) a part-time IT accessibility coordinator was hired; 4) a budget was provided by the CIO for the process; and 5) an action plan was developed by the accessibility subcommittee.

Hokkanen, Record, & Wood (2005) described the steps in their four-year process to Web accessibility: Early, after the accessibility committee was formed, a state IT website for accessibility information was launched. To initiate the process of learning the issues, a summer intern was hired to research issues for compliance and to provide recommendations on how to get started. Based on those recommendations, the state accessibility policy was updated and a usable format for implementation guidelines was created. Accountability and standards were revised. A Web Design Consistency Initiative was launched, in which the Accessibility Committee, the CIO, the Department of Administrative and Financial Services, and a training organization (InforME) collaborated on how to support web design and development.

A Web subcommittee was established to focus on prioritizing and implementing the Web Accessibility Plan recommendations. Implementation was composed of four elements: training, web resources, evaluation and governance, and personnel: 1) For training, the state made a commitment to make basic, updated web design training available to every state employee who worked on state web pages. 2) For resources, webmasters transitioned to a unified software development application.7 Each agency had access to web testing and accessibility tools. They developed a Webmaster Resource Center website and online forum for webmasters.8 3) For evaluation and governance, the state purchased enterprise accessibility evaluation software,9 with which they plan to evaluate web accessibility twice a year. Finally, each agency was required to develop its own internal management plan. 4) For personnel, all agencies were required to appoint a primary web coordinator responsible for compliance with state standards. In addition, a state webmaster directory was developed to facilitate communications and security.

Conclusion

Although the variety of accessibility measures used in this study does not reflect a comprehensive picture of whether these pages are accessible, it does indicate that strength of web accessibility policy alone does not guarantee a practical influence on actual web page accessibility. For important state services, such as locating a job or submitting a tax return, where 75 percent of pages contain barriers for individuals, accessibility is an important issue, especially when access to online services improves citizens' participation in government, in their employment, and in their quality of life. However, improving the situation is complex and requires additional measures. Although developing good accessibility policies is an important step toward more accessible websites, other measures may be needed, such as stronger legislation, clearer and comprehensive policies, better user feedback, consequences for noncompliance, and better education and tools for developers and users.

Hokkanen, Record, & Wood (2005) summarize the benefits of complying with accessibility standards. It improves usability for the public. It provides the state with a consistent, professional look and feel on its websites. The process can provide a consistent branding of all state agency sites. Practices encouraged by the standards (such as using cascading style sheets) ensure compatibility with a variety of browsers and faster download times. Working with the standards tends to produce code that is easier to develop and maintain. And, these issues work toward a higher level of statewide efficiency that reduces costs.

These benefits, along with better access for people with disabilities, improve people's relationship with their state governments and the lives of people with disabilities. When better access translates into better employment opportunities, the state also will benefit in a variety of other ways.

References

Americans with Disabilities Act of 1990, 42 U.S.C. § 12132 (2001).

Blanck, P. D., & Sandler, L. A. (2000). ADA Title III and the Internet: Technology and civil rights. Mental & Physical Disability Law Reporter, 24(5), 855-59.

Bobby WorldWide. (Version 5.0). [Web- and desktop-based computer application]. Wakefield, Mass.: Center for Applied Special Technology. Located at http://bobby.watchfire.com/bobby/html/en/index.jsp.

Coyne, K. P., & Nielsen, J. (2001). Beyond ALT text: Making the Web easy to use for users with disabilities. Fremont, CA: Nielson Norman Group.

Fagan, J. C., & Fagan, B. (2004). An accessibility study of state legislative Web sites. Government Information Quarterly, 21, 65 — 85.

Golden, D. C., & Buck, D. V. (2003). State IT accessibility policy: The landscape of today. Information Technology and Disabilities 9(1). Retrieved Jan. 8, 2004 from http://www.rit.edu/~easi/itd/itdv09n1/golden.htm.

Hokkanen, K., Record, K., & Wood, E. (2005, June 23). The collaborative road to Maine state Web accessibility. Webcast presented by Equal Access to Software and Information (EASI). Archived at http://easi.cc/archive/wood/.

Information Technology Technical Assistance and Training Center. (2004). Overview of state accessibility laws, policies, standards and other resources available on-line. Retrieved Oct. 27, 2004, from http://www.ittatc.org/laws/stateLawAtGlance.cfm.

Klein, D. K., Myhill, W., Hansen, L., Asby, G., Michaelson, S., & Blanck, P. B. 2003. Electronic doors to education: Study of high school website accessibility in Iowa. Behavioral Sciences and the Law, 21(1), 27-49.

National Council on Disability. (1996). Guidance from the graphical user interface (GUI) experience: What GUI teaches about technology access. Washington, D.C.: National Council on Disability.

Rehabilitation Act of 1973, 29 U.S.C. § 701 et seq. (2001)

Rowland, C. (2000). Accessibility of the internet in postsecondary education: Meeting the challenge. Paper presented at Universal Web Accessibility Symposium 2000, Oct. 31, 2000, San Antonio, TX. Retrieved Nov. 16, 2004, from http://www.webaim.org/coordination/articles/meetchallenge.

Schmetzke, A. (2004). Web page accessibility on University of Wisconsin campuses: 2004 survey data and six-year trend data. Web Accessibility Survey Site. Retrieved Aug. 15, 2005, from http://library.uwsp.edu/aschmetz/Accessible/UW-Campuses/Survey2004/contents2004.htm.

Schmetzke, A. (2001). Online distance education — "Anytime, anywhere" but not for everyone. Information Technology and Disabilities 7(2). Retrieved Sept. 10, 2002, from http://www.rit.edu/~easi/itd/itdv07n2/axel.htm.

Sullivan, T., & Matson, R. (2000). Barriers to use: Usability and content accessibility on the Web's most popular sites. Proceedings of the 2000 Conference on Universal Usability (pp. 139 — 144). Arlington, VA, ACM.

Theofanos, M. F., & Redish, J. G. (2003). Guidelines for accessible — and usable — web sites: Observing users who work with screen readers. Interactions, 10(6), 36 — 51.

Thompson, T., Burgstahler, S., & Comden, D. (2003). Research on Web Accessibility in higher education. Information Technology and Disabilities. 9(2), http://www.rit.edu/%7Eeasi/itd/itdv09n2/thompson.htm.

West, D. M. (2004a). Global e-government, 2004. Providence, RI: Brown University. Retrieved Aug. 24, 2005, from http://www.insidepolitics.org/egovt04int.pdf.

West, D. M. (2004b). State and Federal e-government in the United States, 2004. Providence, RI: Brown University. Retrieved Aug. 24, 2005, from http://www.insidepolitics.org/egovt04us.pdf.

Yu, H. (2002). Web accessibility and the law: Recommendations for implementation. Library Hi Tech, 20(4), 406 — 419.

Zeng, X., & Parmanto, B. (2004). Web content accessibility of consumer health information web sites for people with disabilities: A cross sectional evaluation. Journal of Medical Internet Research. 6(2), e19.

Endnotes

1As of this writing, version 2 of the WCAG is in development and is expected to go into effect soon. This version is expected to eliminate the priorities in favor of another organizing principle. However, the three priorities will be useful for some time because they have served as a standard for the discussion and research to date.

2 Version 2 of the WCAG will theoretically be developed to encompass languages such as JavaScript, VB Script, Java, Flash, and other Web-based technologies.

3 The ITTATC State Policy database is located at http://www.ittatc.org/laws/stateLawAtGlance.cfm.

4 The exact language of the feedback from Bobby on failure for this guideline is "Provide alternative text for all images."

5 The exact language of the feedback from Bobby on failure for this guideline is "Use relative sizing and positioning (% values) rather than absolute (pixels)."

6 The exact language of the feedback from Bobby on failure for these guidelines are "Identify the language of the text," "Provide a summary for tables," and "Separate adjacent links with more than whitespace."

7 Maine chose Macromedia Dreamweaver as the web development software.

8 HiSoftware (Hiawatha Island Software, Inc.) accessibility tools AccVerity/AccRepair, Hi-Caption, and Link Checker were chosen for accessibility testing software.

9 HiSoftware evaluation tool AccMonitor was used.