Sandra Sanchez-Gordon, Sergio Luján-Mora. Journal of Universal Computer Science (J.UCS), 22(1), p. 55-81. ISSN 0948-695X. ISSN online: 0948-6968.
Massive Open Online Courses (MOOCs) have great potential to provide learning opportunities for people around the world. However, to reach their full potential, MOOCs need to meet the accessibility needs of diverse learners, with and without disabilities. In the literature review, we have found some published research on accessibility evaluations of MOOCs content and platforms, but we have not found published research on how to design existing and future MOOC platforms to assist authors in producing accessible content. The main purpose of this research is to contribute to the discussion about the future of inclusive online learning, by proposing a software design to incorporate features in MOOC platforms to enable, support and guide authors toward conceptualizing, designing, building and testing accessible MOOCs. We also present the results of an evaluation of the accessibility issues of Studio, the edX course-authoring software, based on ATAG 2.0.
Keywords: ATAG, MOOC, Massive Open Online Courses, UAAG, WCAG, WCAG-EM, WGAC2ICT, accessibility, accessibility conformance evaluation, disabilities, edX, mobile learning, online learning
From the educational perspective, a Massive Open Online Course (MOOC) is an online course with no formal entry requirements, no participation limits and is free of charge [Gaebel, 13]. In 2013, Sanchez-Gordon and Luján-Mora  stated, “MOOCs unique potential lies in the fact that they have discovered that there are millions of people of all ages around the world eager to learn”. In the words of Agarwal, founder of the U.S. based MOOC platform edX, “the past few centuries have witnessed revolutions in virtually every area of our world – health, transport, communications and genomics, to name but a few. But not in education. Until now, that is, with the advent of MOOCs (..) One way MOOCs have changed education is by increasing access. MOOCs make education borderless, gender-blind, race-blind, class-blind and bank account-blind. MOOCs are democratizing education” [Agarwal, 13]. However, Agarwal failed to point out that MOOCs could also make education disability-blind, which is the ultimate goal of this research.
Due to its open and massive nature, a MOOC can offer a learning experience to thousands – even hundreds of thousands – of diverse users at the same time. That is the reason why MOOCs are a great opportunity for people of all conditions and capabilities. Traditional face-to-face educational environments often have many accessibility problems, especially in developing countries. For example, a blind student cannot see their teacher’s presentation or writing on the blackboard; therefore, they have difficulty in following a class. Similarly, a deaf student cannot hear the teacher's explanations or interact with peers. With the use of the right approaches, these barriers may disappear in online learning environments in general and MOOCs in particular.
One important benefit of having an accessible MOOC is that the situation of a learner with disabilities may go completely unnoticed because the communication is mediated by technology, hence the person can be treated truly equally. This will guarantee real inclusiveness, which is not possible in a face-to-face environment, unfortunately.
However, to achieve inclusiveness, MOOCs must meet the accessibility needs of diverse learners. In the literature review, we have found some published research on accessibility evaluations of MOOCs content and platforms [Al-Mouh et al., 14; Bohnsack and Puhl, 14; Calle-Jimenez et al., 14; Horton and Sloan, 14; Iniesto and Covadonga, 14; Singleton, 13; Young, 14]. From 2015, there is published research on holistic proposals based on existing standards for providing accessible services for MOOC students [Iniesto and Covadonga, 15; Rodríguez-Ascaso and Boticario, 15]. Nevertheless, we have not found published research on how to design existing and future MOOC platforms to assist authors in producing accessible content. We think this is due to two main reasons. First, there is the relative novelty of the MOOCs phenomena. Second, there is a need to identify accessibility issues in current MOOCs, obtain hands-on experience in solving them in particular instances and reflect on the problem and its main causes as previous steps to envision a generic architecture for accessible MOOCs.
The purpose of this paper is twofold. On the one hand, to contribute to a better understanding of the accessibility challenges faced by learners with disabilities using MOOCs. On the other hand, to contribute to the discussion about the future of inclusive online learning, by proposing a software design to incorporate features in MOOC platforms to enable, support and guide authors toward conceptualizing, designing, building and testing accessible MOOCs. In this research, we use the MOOC platform edX as a case study. Currently, edX is one of the main MOOC platforms used worldwide; it has an open source version called Open edX but it is facing legal action in the U.S. due to accessibility problems.
This paper is organized as follows: Section 1 characterizes MOOCs, learners with disabilities, accessibility in the context of MOOCs and accessible mobile learning with MOOCs. Section 2 presents the results of a conformance evaluation of Studio, the edX course-authoring software, for ATAG 2.0. Section 3 presents the proposal of a design for improving the accessibility of MOOCs created and published in the edX platform. Finally, Section 4 concludes the paper with final remarks and a discussion about future work.
The ancestors of MOOCs are distance learning, electronic learning or e-learning, and online learning. We can trace back the history of distance learning as far as 1728, when Caleb Philips announced in the Gazeta Boston, a course in shorthand by correspondence [Sanchez-Gordon and Luján-Mora, 14a]. Similarly, the beginning of e-learning can be found in the 1960s with the Programmed Logic for Automatic Teaching Operations (PLATO). PLATO was developed at the University of Illinois to offer coursework to the university´s students, local schools and other universities [Gentile, 67]. The development of online learning technology began in the 1990s with the creation of learning management systems (LMS). A first attempt to use the web and the open nature of the internet in education dates from 2003. David Wiley, from Utah State University, experimented with the use of blogs and aggregators to support distributed communications among students [Martindale and Wiley, 04].
In 2008, George Siemens and Stephen Downes taught the course “Connectivism and Connective Knowledge” at the University of Manitoba, Canada. This course had 2,200 online students, in addition to 24 face-to-face students. This approach involved not only a large number of students taking the course at the same time using the internet, but also distributing the learning environment over several online spaces (i.e. wikis, blogs), giving participants the freedom to create those spaces and contribute with new content, whilst aggregating these massive contributions from learners to the course. Dave Cormier  and Bryan Alexander  coined the term MOOCs to refer to this new form of online learning, deriving the term from the concept of MMOGs – Massively Multiplayer Online Games. Currently, the type of online courses based on this connectivist, distributed, peer-learning model is called cMOOC.
In 2011, Sebastian Thrun and Peter Norving ran the first MOOC definitely massive, the course “Introduction to Artificial Intelligence” through Stanford University, U.S. This course had 160,000 students registered. The need for architecture to support this level of massiveness led to the creation of specific MOOC platforms. The main challenges were large-scale content delivery and learning assessment. This approach returned to the concept of a learning environment as one central meeting point. However, even in these more controlled online courses, learners find ways to spread their interactions into complementary online spaces. This type of online course is called xMOOC [Sanchez-Gordon and Luján-Mora, 14a].
Since its inception, the MOOCs phenomenon has grown steadily. According to Shah , around 4,180 courses will have begun by March 2016, as illustrated in Figure 1. This contrasts with 2,400 courses that were available by July 2015. That is, the number of courses almost doubled in less than a year.
As for the intended audience, at the beginning, MOOCs’ scope focus was in providing introductory university level courses. Nevertheless, MOOCs rapidly expanded their scope from higher education to other dimensions, such as life-long learning [Liyanagunawardena and Williams, 16], corporate training [Sanchez-Gordon et al., 15b; Sanchez-Gordon and Luján-Mora, 15c], secondary and primary education [Cairneagle Associates, 14].
Coursera, the first MOOC platform, was launched in 2012. Since then, several MOOC platforms have appeared worldwide. By the end of 2015, Coursera was the lead platform with 17 million users, followed by edX with 5 million users, Udacity with around 4 million users and FutureLearn with almost 3 million students. Coursera, edX and Udacity are U.S. based, while FutureLearn is from the United Kingdom. Besides FutureLearn, there are other important region-based MOOC platforms, such as MiriadaX (Spain) with 1 million users, GouKr (China) with 800,000 users, OpenClassRooms (France) with 650,000 users, Open2Study (Australia) with 450,000 users, Iversity (Germany) with 600,000 users, Rwaq (Saudi Arabia) with 300,000 users and FUN (France) with 250,000 users. According to Class Central data, more than 35 million students had signed up for at least one MOOC by the end of 2015 [Shah, 15].
In a previous work, we categorized different variants of MOOCs, as summarized in Table 1 [Sanchez-Gordon and Luján-Mora, 14a]. Some of these variants deviate considerably from the original concept but recognize its origin. It is expected that MOOCs will continue to evolve. As in the natural selection of species, only the strongest variants will survive. For the strongest, we refer to those MOOCs variants that will manage to achieve the goal of providing quality online learning with a sustainable business model.
Given the ample spectrum of variants, it is difficult to come up with a straightforward definition of MOOC. Vipond and Clarey  highlight the most important features, explaining that “massive open online courses are online courses and collections of learning content aimed at unlimited participation and open Internet access. In addition to traditional course materials such as videos, reading assignments, exercises, and other reference materials, MOOCs can also be used to enhance interactions and create collaborative opportunities for instructors and learners.”
According to the World Health Organization, more than one billion people live with some form of disability [WHO, 11]. That is, almost one-fifth of the global population has some kind of disability. This fact makes this community the world’s largest minority. The same organization stated, “Lower income countries have a higher prevalence of disability than higher income countries” [WHO, 11]. Unfortunately, the greater the poverty, the greater the exclusion produced by the presence of a disability.
There are several types of disabilities, including those that can be a permanent condition from birth or acquired later in life:
The number of persons with disabilities increases appreciably if we take into account not only permanent disabilities. This number should also include people with the following conditions [Sanchez-Gordon and Luján-Mora, 14b]:
All of the above makes us ponder that we – or someone close to us – could potentially be in a disability situation at some stage of life, but we do not foresee this due to our limited understanding of disability. Of course, any disability situation also affects us as learners.
According to the United Nations Convention on the Rights of Persons with Disabilities (CRPD) “technology design should take into account accessibility and usability features for the protection and promotion of the human rights of persons with disabilities” [United Nations, 08]. Of particular importance in the context of this research is Article 24 of the CRPD, which recognizes the right to education. CRPD signatories “must make sure that persons with disabilities are able to get access not only to general education but also to tertiary education, vocational training, adult education and lifelong learning without discrimination and on an equal basis with others” [United Nations, 08].
In 2009, the U.S. Government Accountability Office (GAO) conducted a study that found that although the number of students with disabilities who pursue higher education was increasing, barriers still existed. As a result, students with disabilities participate in and complete higher education at lower rates than students without disabilities do [GAO, 09].
In Europe, Morales  reports that in Spain only 3.6% of the population with disabilities completes higher education while the correspondent percentage for the general population is 20%. Besides, 84% of Spain’s college students with disabilities state that they face several barriers through their studies.
In Latin America, Molina  presents the following data from Colombia: only 2.3% of the population with disabilities has some level of higher education (technical, technological, or professional), 1% complete their higher education and only 0.1% obtain graduate degrees.
That is, higher education penetration among the population with disabilities has a long way to go before reaching similar levels than the general population. Part of the problem is that higher education institutions might not have had accessibility in mind when building facilities and getting equipped. Providing accommodation for students with disabilities in existing facilities and equipment is costly and might be not feasible.
The International Organization for Standardization (ISO) defines accessibility as, “The usability of a product, service, environment or facility by people with the widest range of capabilities” [ISO, 12].
The Office for Civil Rights (OCR) of the U.S. Department of Education explains: “Accessible means a person with a disability is afforded the opportunity to acquire the same information, engage in the same interactions, and enjoy the same services as a person without a disability in an equally effective and equally integrated manner, with substantially equivalent ease of use. The person with a disability must be able to obtain the information as fully, equally and independently as a person without a disability” [OCR, 13].
Tim Berners-Lee, leader of the World Wide Web Consortium (W3C), stated, “Accessibility is the art of ensuring that, to as large an extent as possible, facilities (such as, for example, web access) are available to people whether or not they have impairments of one sort or another” [W3C, 99].
In 1997, the W3C created the Web Accessibility Initiative (WAI) with the aim of studying the problems of accessibility to the web, developing guidelines and providing resources [W3C, 09]. The WAI is an international authority on web accessibility. WAI has developed three sets of guidelines [W3C, 14a]:
WCAG, ATAG and UAAG work together to make the World Wide Web accessible, as shown in Figure 2. Each set of guidelines is aimed at one of three main elements: web content in the middle, authoring tools to create that content and user agents to consume the content.
Smith  gives the following example to illustrate the combination of the three sets of guidelines for alternative text on images:
Up to January 2016, Twitter is an example of an authoring tool that is not fully compliant with ATAG: it does not allow authors to include alternative texts to the images in a tweet. Figure 3 shows that the interface only enables the author to choose a file. As a result, tweets in the user timeline do not have alternative texts (i.e. are not fully compliant with WCAG), and blind users who use a screen reader cannot perceive images in those tweets (i.e. are not fully compliant with UAAG).
In contrast, Figure 4 shows that Studio, the edX course-authoring software, does enable authors to insert an alternative text (named “Image description”) for an image. However, in Studio it is not mandatory to provide this alternative text. Beside, Studio does not verify that images contain alternative text.
The versions of the three sets of guidelines as of January 2016 are WCAG 2.0, released in 2008 as a W3C Recommendation. It was recognized in 2012 as the standard ISO/IEC 40500. ATAG 2.0 released in 2015 as a W3C Recommendation. UAAG 2.0 released in 2015 as a W3C Working Group Note.
Besides, in 2013, W3C released the “Guidance on Applying WCAG 2.0 to Non- Web Information and Communications Technologies (WCAG2ICT)” as a W3C Working Group Note. WCAG2ICT enables the use of WCAG 2.0 principles, guidelines and success criteria for non-web content and software, including mobile applications [W3C, 13].
In addition, in 2014, W3C released the “Website Accessibility Conformance Evaluation Methodology 1.0 (WCAG-EM)” as a Working Group Note. The WCAGEM defines a procedure to evaluate how well websites conform to the WCAG [W3C, 14b].
A study made in the European Higher Education Area (EHEA) about standards and indicators for disability describes 31 indicators to accommodate the accessibility needs of students with disabilities, [Diez et al., 11], among them:
Besides physical barriers, there are also attitudinal and communication barriers with instructors, fellow students and administrative personnel on college campuses. People often do not know how to interact and communicate with persons with disabilities, and thus prefer to avoid contact. Finally, to accommodate students with disabilities, the university has to put in place adaptations in policies, procedures, curricula and learning/assessment activities.
In the context of MOOCs, due to its massiveness and openness, there is a potential for great diversity among learners. Because of that, MOOCs are a good alternative for providing quality online learning to persons with disabilities. For example, Sanchez-Gordon and Luján-Mora  explain: “MOOCs bring great opportunities to enhance the quality of life of older people by enabling lifelong learning and inclusion in learning communities”. This favours cognitive stimulation, a sense of belonging and social engagement. However, the same authors point out that “MOOCs can present some accessibility barriers that could hamper full participation by elderly students” [Sanchez-Gordon and Luján-Mora, 13]. In addition, non-native speakers face barriers when taking MOOCs due to their level of proficiency in the course language, which turns them into learners with cognitive problems [Sanchez- Gordon and Luján-Mora, 15d]. This diversity requires satisfying the accessibility needs of learners with different types of disabilities [Sanchez-Gordon and Luján- Mora, 14b].
Nevertheless, it is feasible to modify existing MOOCs platforms to improve their capacity to support the creation and publication of accessible content. In addition, it is even easier to create new MOOC platforms as accessible by design.
Singleton  wrote the first academic research in the field of accessibility in MOOCs. This author stated, “There is absolutely no research on MOOCs and how they have or will impact individuals with disabilities. The phenomenon is still too new and, as such, this area of research has not been developed yet.” This study explained that to evaluate the accessibility of MOOCs, it is necessary to look at research on how learners with disabilities have been impacted upon by previous online learning technologies. Some of the issues pointed out by this study are the lack of accessibility of the course management tools and the course materials, like PDF documents and slide presentations, as well as the lack of technological support at an institutional level, to resolve any challenges that the students with disabilities encountered.
Horton and Sloan  explained that when usability is envisioned in terms of a large and diverse population including persons with disabilities, as is the case in MOOCs, it becomes accessibility.
Young  pointed out the importance of usability in MOOC design. This author explained that by questioning, investigating and ultimately testing the usability of course materials in MOOCs, the learning experience would be more accessible. Unfortunately, simply encouraging usability testing of course materials and course design will not solve usability issues in MOOCs. Instructors should present learning materials in formats – other than text – that are appropriate to their courses’ purpose, audience and medium.
Bohnsack and Puhl  evaluated the accessibility of five MOOC platforms: Udacity, Coursera, edX, OpenCourseWorld and Iversity. Their study determined that none of the evaluated MOOC platforms was fully accessible for blind users, especially in the initial stages of interaction. These authors concluded that accessibility was not in focus when the platforms were built.
Iniesto and Covadonga  made an accessibility analysis of three MOOC platforms in Spanish; MiriadaX, COLMENIA, and UNED COMA. These authors made an analysis based on automated accessibility evaluation tools, disability simulators, automated testing tools oriented to blind users and content analysis. These authors found that all three platforms had serious accessibility problems at both platform and content level.
Al-Mouh et al.  conducted an experiment that included a heuristic evaluation of the level of accessibility of ten Coursera’s MOOCs. All the evaluated MOOCs failed to comply with WCAG 2.0.
Regarding accessibility issues on specific-purpose MOOCs, Calle-Jimenez et al.  elaborated on the challenges to produce accessible geo-MOOCs, i.e. MOOCs on Geographical Information Systems (GIS). This kind of MOOCs, by nature, has inherent problems of accessibility of geographical content for learners with visual disabilities.
Rodríguez-Ascaso and Boticario  differentiated between accessibility in the MOOC platform and accessibility of the MOOC content. These authors proposed a holistic perspective with an open and standards-based architecture of services to attend students with and without disabilities, which can be integrated into different learning platforms. This proposal was evaluated in the context of the “European Unified Approach for Accessible Lifelong Learning (EU4ALL)” project [EU4ALL, 13].
Iniesto and Covadonga  proposed a strategy to make MOOCs more accessible by using metadata to analyze the characteristics of the educational resources, according to the needs and preferences of the users using the “IMS Access for All (AFA) v3.0” specification to profile both the “personal needs and preferences (PNP)” and the “digital resource descriptions (DRD)”. The goal is to design a recommender system for MOOCs adapted to the user’s needs.
Despite these research efforts, for now, MOOCs are not accessible enough. Hence, the promise of democratizing education and giving access to education to everyone has yet to be fulfilled. It is necessary to analyse the problem and propose solutions to improve the inclusion of diverse learners – both able and disabled – in MOOCs platforms and content [Sanchez-Gordon and Luján-Mora, 16].
In recent years, the growth in the use of mobile devices (mainly smartphones and tablets) and the cloud, and native apps that are executed on them, has exploded [Alamri et al., 14]. The rate of growth is so fast that Deloitte estimated that about one billion people transitioned from regular phones to smartphones during 2015 [Lohman, 15].
In 2012, a study highlighted that 70.4% of the respondents revealed that they accessed MOOC course materials using mobile devices, the main reason being location flexibility (61.3%), and the main resources accessed were group discussions (65.9%) [De Waard et al., 12]. At that time, the potential of the merger of mobile learning and MOOCs was already evident.
In 2013, Coursera and Udemy were the first MOOC platforms to launch mobile apps. In 2014, Udemy reported that 20% of its course content consumption was via mobile, and it went up to 40% in corporate training. In December 2015, edX released version 2.0.2 of its mobile app for iOS and version 1.0.09 for Android, which allows for account registration, course enrolments, watching and downloading videos, accessing announcements and content. In January 2016, Coursera released version 2.1.2 of its mobile app for iOS and version 22.214.171.124 for Android, which allows users to search for courses, enrol, watch and download videos or use audio-only mode, and receive notifications.
There is a strong relationship between mobiles and accessibility. According to WAI, mobile accessibility refers to “making websites and applications more accessible to people with disabilities when they are using mobile devices” [W3C, 15d]. The W3C’s Mobile Accessibility Task Force work includes the creation of mobile techniques for WCAG and the development of design guidance and mobile web accessibility best practices. On February 2015, this task force released the first working draft of “Mobile Accessibility: How WCAG 2.0 and Other W3C/WAI Guidelines Apply to Mobile”, which highlights the relevance of UAAG 2.0 in the mobile context [W3C, 15e]. Nevertheless, until a mobile accessibility standard emerges, it is recommended to use the WCAG2ICT [W3C, 13].
In 2014, Nakata  differed with the W3C approach and pointed out that “accessibility for mobile apps is relatively new and complicated.” This author explained that a mobile app is a different piece of software to a browser. Hence, mobile development platforms have functionality that allow developers to implement specific-purpose accessibility APIs and take advantage of the accessibility features of both mobile operating systems and devices. According to this author, “accessibility APIs would also provide a perfect meeting point between software and assistive technology”, Nakata .
This section presents an evaluation of accessibility using the MOOC platform edX as a case study. First, we describe the method and then we present the results and discussion. Due to space limitations, we are able to show only an extract of the results.
edX is a nonprofit, U.S. based, MOOC platform. The Massachusetts Institute of Technology (MIT) and Harvard University founded it in 2012. In 2013, edX released its platform software as an open source project, creating Open edX so developers worldwide can build enhancements [edX, 15a]. Several organizations worldwide have adopted Open edX to launch their own MOOC initiatives [gitHub, 15]. According to Shah , Open edX is becoming the “de facto platform for organizations and groups who are looking to host their own MOOCs.”
In February 2015, the U.S. National Association of Deaf (NAD) filed two federal lawsuits against Harvard and MIT respectively, alleging both universities “had violated U.S. antidiscrimination laws by failing to provide closed captioning in their online lectures, courses, podcasts and other educational materials” [NAD, 15]. The bottom line is that these lawsuits are targeted mainly at video content. WCAG 2.0 at Level A requires captioning of recorded video and WCAG 2.0 at Level AA requires captioning of live events. If these lawsuits are successful, and Harvard and MIT are legally required to caption all MOOC video content, other MOOC providers and campuses will follow. The interesting news is that captioning video benefits users with hearing disabilities and other users looking for particular portions of videos, since it enables rapid search functionality [NAD, 15; Ferris, 15].
In April 2015, edX Inc. entered into a voluntary settlement agreement with the U.S. Department of Justice (DOJ), requiring edX to “take steps to ensure the courses it distributes online meet certain minimal accessibility standards” [DOJ, 15; edX, 15b]. With this agreement, edX avoids a lengthy and costly public judicial process. The agreement calls for [edX, 15b]:
According to the agreement, in achieving conformance, edX may rely upon the UAAG, ATAG and WCAG2ICT standards, as well as guidance published by the W3C’s Mobile Accessibility Task Force.
In this regard, edX stated, “At edX, we seek to understand and respect the unique needs and perspectives of the edX global community. We value every learner, and are committed to being a leader in expanding access to all, including learners with disabilities” [edX, 15b]. In addition, the edX’s Accessibility Best Practices for Developing Course Content stated: “EdX is dedicated to creating a platform that is not only itself accessible, but also enables course creators to create accessible content” [edX, 15c].
However, there is still a road ahead since there are several accessibility issues in the edX platform, as we show in the following sections.
The study presented in this paper aims to evaluate the conformance of Studio, the edX course-authoring software, for ATAG 2.0. Details regarding the study design follow.
The product evaluated is the version of edX Studio available at the edX public sandbox at https://studio.sandbox.edx.org/.
One issue with evaluations is the level of independence of the evaluator. It is common that when evaluations are solicited or financed by the owners of the evaluated product or service, the results tend to have a bias. In this case, the evaluators belong to the academy and they do not have any relationship with edX Inc. Hence, there is independence in the evaluation presented in this paper.
First, an accessibility expert with 23 years of professional experience carried out the independent conformance evaluation of edX Studio for ATAG 2.0. After that, a second accessibility expert, with 16 years of professional experience, checked the results. Both evaluators have a deep knowledge of both web accessibility and MOOCs.
ATAG 2.0 has two parts. Part A addresses the problem of making the authoring-tool’s user-interface accessible. Part B addresses the problem of enabling the authoring tool to give support and guidance in the production of content with a good level of accessibility. Under each part, there are four high-level principles that contain sets of guidelines. For each guideline, testable success criteria are provided.
The conformance evaluation presented in this paper covers ATAG 2.0 Part B Principle B.2, which answers the question: how do I make my content authoring software enable, support and guide authors towards producing content that is more accessible to users with disabilities?
ATAG 2.0 Part B establishes four principles [W3C, 15a]: “Fully automated processes produce accessible content (B.1), support authors in the production of accessible content (B.2), support authors in improving the accessibility of existing content (B.3), and authoring tools promote the use of their accessibility features (B.4)”.
From them, we selected principle B.2 because enabling authors for the production of accessible content is the basis for improving the accessibility of MOOCs. This principle contains five guidelines:
Due to space limitations, we present the results of the conformance evaluation for Guidelines B.2.1, B.2.2, and B.2.3.
For the evaluation, the evaluator used the default account provided by edX Studio public sandbox to login as staff and proceeded to apply WCAG-EM. The evaluator test-drove features by re-creating a real author scenario where an author creates a course, with sections, subsections and units. Then, the evaluator proceeded to add images, videos, text content, quizzes and so on, while evaluating the conformance with the success criteria.
Tables 2, 3 and 4 present the results of the conformance evaluation for guidelines B.2.1, B.2.2., and B.2.3. For each ATAG 2.0 success criterion, the column W3C’s Description presents the official definition [W3C, 15a], the column Disabilities lists the main disabilities (visual, hearing, motor, and cognitive) related to the criterion, the column C indicates whether there is compliance with the criterion or not (Yes, No), and the column Rationale gives a detailed explanation.
In summary, the main findings of the conformance evaluation results presented are:
In addition, edX Studio does not provide enough documentation or user interface features to guide and support the authors about how to address accessibility problems that the tool itself has and the generated course content might have.
There are many improvements that edX developers should implement in Studio to achieve compliance with levels A and AA of ATAG 2.0. This is necessary to honour the agreement made with the U.S. Department of Justice.
This section presents a design proposal to improve the accessibility of MOOCs created and hosted in the edX platform for the benefit of diverse learners. We hope to contribute to the efforts towards seamless inclusive online learning. Currently, there are accessibility features in most operating systems and browsers, as well as assistive technologies for several disabilities. Nevertheless, many of them require users to explicitly invoke and configure them. Sloan et al.  warned about the potential negative psychological effects “caused by the introduction of an assistive technology that dramatically changes how a person interacts with a computer may lead to rejection of that technology” [Sloan et al., 10].
Figure 5 shows the proposed design, taking into account the architecture of edX and the findings of our ATAG conformance evaluation for Studio. The design has two components: a user-side component and an author-side component.
The user-side component has four layers and one Accessibility Application Program Interface (API). Online learners might or might not have disabilities. Learners with disabilities might use several assistive technologies, depending on their accessibility needs and preferences. For example, a blind learner could use a screen reader while a deaf user could use a speech-to-text tool. Hence, learners interact either with the first layer (assistive technology) or directly with the second layer (browser). Both layers must be compliant with UAAG. The accessibility API is necessary to establish communication between the first layer, the second layer and the operating system.
The third layer (the Adaptive Content Extension) is the layer we propose to include as an extension of the current edX architecture. This layer adds features to the user interfaces of the fourth layer, the edX Learning Management System (LMS).
The third layer contains an adaptation engine and three persistence subcomponents; the user accessibility profiles’ database, the resource profiles’ database and the adaptation rule set database. The user accessibility profiles’ database stores the accessibility preferences of the registered users. The resource profile database stores accessibility-related information of the content resources. The adaptation rule set database stores logic that is executed by the adaptation engine according to the user accessibility preferences [Sanchez-Gordon and Luján-Mora, 15a]. The rules are defined using eXtended Markup Language (XML).
As an example of a rule, Figure 6 shows an extract of the adaption rule sequence to be executed if the user accessibility preferences’ profile indicates dyslexia.
This adaption rule sequence is based upon the research on dyslexia, developed by De Santana et al.  and Kalyvioti and Mikropoulos  regarding the text font, text size, text justification and line spacing that best suits people with dyslexia.
According to this adaptation rule sequence, the text font changes either to Verdana, Arial or Calibri, depending on the first text font found. If none of these text fonts is present, any sans serif font is used. Then, the text size is set to 12 points, the line spacing is set to two lines and the text is unjustified. Figure 7 shows an extract of a sample’s content, before and after the adaptation.
In summary, adaptive content presentation involves personalizing the contents delivered to the user to enhance their accessibility [Stephanidis et al., 98]. The adaptation engine access to the content store database to fetch and present the appropriate content resources adapted to the accessibility preferences of the user. The resources in the content store database must be compliant with WCAG.
This design allows MOOC learners to manage their accessibility user profile by selecting a combination of accessibility issues that best suit their current life situation. The use of an accessibility user profile represents an improvement, compared with current approaches used in websites and web applications, where the user must manually configure specific technical adaptations, such as colour combinations or text fonts. Figure 8 shows the user interface for learners to select accessibility preferences: the learner only has to select their disabilities and the platform automatically applies the best adaptations. In addition, the Accessibility Preferences user-interface has an “Advanced Options” feature that provides well-informed users with the freedom to select specific adaptations, if desired [Sanchez-Gordon and Luján-Mora, 16].
The author-side component also has four layers and one Accessibility Application Program Interface (API). Similar to learners, authors also might or might not have disabilities. Authors with disabilities might use assistive technologies depending on their accessibility needs and preferences. Hence, authors also interact, either with the first layer (assistive technology) or directly with the second layer (browser). Both layers must be compliant with UAAG. The accessibility API is necessary to establish communication between the first layer, the second layer and the operating system.
The third layer (the Adaptive Content Extension) is the layer we propose to include in the author-side as an extension of the current edX architecture. This layer adds features to the user-interfaces of the fourth layer, the edX Studio, to make it comply with ATAG.
The third layer contains an adaptive content manager that guides the author through the creation of accessible content resources, warns the author when accessibility-related information is not provided and performs accessibility checks on the created content. In addition, the adaptive content manager keeps the resource profile database updated and enables the author to store several alternative formats for the same content in the content store database, e.g. video lessons with subtitles, video transcripts and accessible PDF files. These contents must also be WCAG compliant to guarantee the accessibility of the MOOC.
Software tools and educational content must adapt to users, not the other way round. It is essential to understand the needs and preferences of persons with disabilities and the ways they interact with computers and mobile devices, to design MOOC platforms and contents that decrease accessibility barriers. In this research, we proposed a design that will allow MOOC content authors to create accessible content for the benefit of learners, with and without disabilities.
The authoring software provided by a MOOC platform must have features that enable, support and guide the creation of accessible MOOC content. Ideally, accessibility should be so integrated into the course-authoring tool that authors who produce content with it would be producing accessible content without having to specifically be experts upon it or even having to think about it.
Strong accessibility by design can ultimately benefit all users of a software system. ATAG provides guidelines to design accessible authoring tools that can be applied to MOOC platforms.
Continuing research is essential for increasing the accessibility of MOOCs and improving the inclusion of diverse learners in MOOCs. As Howard Rosenblum, CEO of the U.S. National Association of the Deaf, stated, “Online content represents the next frontier for learning and lifelong education” [NAD, 15].
For immediate future work, we plan to complement the conformance evaluation presented in this study for all ATAG 2.0 Part B principles. In addition, we plan to build and test a prototype of the design proposed in this paper.
Extended future work includes conformance evaluation of user agents (such as browsers, screen readers, multimedia players) using UAAG 2.0, as well as MOOC accessibility in the mobile context (including mobile web content and mobile apps).
This work has been partially supported by the Prometeo Project by SENESCYT, Ecuadorian Government.
[Alamri et al., 14] Alamri, A., Muhammad, G., Al Elaiwi, A., Al-Mutib, K., Hossain, M.: “Media Content Adaptation Framework for Technology Enhanced Mobile e-Learning”. Journal of Universal Computer Science, 20(15), 2016-2023. 2014.
[Alexander, 08] Alexander, B.: Connectivism course draws night, or behold the MOOC. 2008. http://infocult.typepad.com/infocult/2008/07/connectivism-course-draws-night-or-behold-themooc. html
[Al-Mouh et al., 14] Al-Mouh, N., Al-Khalifa, A., Al-Khalifa, H.: “A first look into MOOCs accessibility”. In Proc. International Conference on Computers Helping People with Special Needs, 145-152. Springer International Publishing. 2014.
[AG, 10] Australian Government: “Web Accessibility National Transition Strategy”. 2010. http://www.finance.gov.au/publications/wcag-2-implementation/docs/wcag-transitionstrategy. pdf
[Agarwal, 13] Agarwal, A.: “Online universities: It is time for teachers to join the revolution”. The Guardian. 2013. http://www.theguardian.com/education/2013/jun/15/university-education-online-mooc
[Bohnsack and Puhl, 14] Bohnsack, M., Puhl, S.: “Accessibility of MOOCs”. In Proc. International Conference on Computers Helping People with Special Needs, 141-144. Springer International Publishing. 2014.
[Cairneagle Associates, 14] Cairneagle Associates: “MOOCs: opportunities for their use in compulsory-age education”. 2014. https://www.gov.uk/government/publications/moocs-opportunities-for-their-use-incompulsory- age-education
[Calle-Jimenez et al., 14] Calle-Jimenez, T., Sanchez-Gordon, S., Luján-Mora, S.: “Web accessibility evaluation of massive open online courses on geographical information systems”. In Proc. IEEE Global Engineering Education Conference, 680-686. 2014.
[Cormier, 08] Cormier, D.: “The CCK08 MOOC – connectivism course, 1/4 way”. 2008. http://davecormier.com/edblog/2008/10/02/the-cck08-mooc-connectivism-course-14-way/
[De Waard et al., 12] De Waard, I., Koutropoulos, A., Hogue, R., Abajian, S., Keskin, N., Rodriguez, C., Gallagher, M.: “Merging MOOC and mLearning for increased learner interactions”. International Journal of Mobile and Blended Learning, 4(4), 34-46. 2012.
[De Santana et al., 12] De Santana, V., De Oliveira, R., Almeida, L., Baranauskas, M.: “Web accessibility and people with dyslexia: a survey on techniques and guidelines”. In Proc. International Cross-Disciplinary Conference on Web Accessibility. 35-47. 2012.
[Diez et al., 11] Diez, E., Alonso, A., Verdugo, M.A., Campo, M., Sancho, I., Sánchez, S.: “European Higher Education Area: standards and indicators of good practices for the attention of college students with disabilities”. 2011. http://sid.usal.es/libros/discapacidad/26032/8-1/espacio-europeo-de-educacionsuperior.aspx
[DOJ, 15] U.S. Department of Justice: “Settlement agreement between the United States of America and edX Inc. under the Americans with Disabilities Act”. 2015. http://www.justice.gov/sites/default/files/opa/pressreleases/ attachments/2015/04/02/edx_settlement_agreement.pdf
[edX, 15a] edX: “The Open edX”. 2015. http://code.edx.org/
[edX, 15b] edX: “edX adopted the following website accessibility policy”. 2015. https://www.edx.org/accessibility
[edX, 15c] edX: “Accessibility best practices for developing course content”. 2015. http://edx.readthedocs.org/projects/edx-partner-coursestaff/ en/latest/accessibility/best_practices_course_content_dev.html
[EU4ALL, 13] EU4ALL: “European Unified Approach for Accessible Lifelong Learning”. 2013. http://eu4all-project.atosresearch.eu
[Ferris, 15] Ferris, L.: “Thoughts on the MOOC captioning lawsuit”. 2015. http://blog.kaltura.org/thoughts-mooc-captioning-lawsuit/
[Gaebel, 13] Gaebel, M.: “MOOCs massive open online courses”. European University Association. 2013.
[GAO, 09] U.S. Government Accountability Office: “Higher education and disability, education needs a coordinated approach to improve its assistance to schools”. 2009. http://www.gao.gov/new.items/d1033.pdf
[Gentile, 67] Gentile, J.: “The first generation of computer-assisted instructional systems: an evaluative review”. AV Communication Review, 15(1), 23-53. 1967.
[Githens, 07] Githens, R.: “Older adults and e-learning: opportunities and barriers”. The Quarterly Review of Distance Education, 8(4), 329–338. 2007.
[gitHub, 15] gitHub: “Sites powered by Open edX”. 2015. https://github.com/edx/edx-platform/wiki/Sites-powered-by-Open-edX
[GOC, 11] Government of Canada: “Standard on web accessibility”. 2011. http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=23601
[Horton and Sloan, 14] Horton, S., Sloan, D.: “Accessibility in practice: a process-driven approach to accessibility”. In Langdon, P., Lazar, J., Heylighen, A., Dong, H. (Eds.), Inclusive designing – joining usability, accessibility, and inclusion, 105-115. Springer International Publishing. 2014.
[Iniesto and Covadonga, 14] Iniesto, F., Covadonga, R.: “Accessibility assessment of MOOC platforms in Spanish: UNED COMA, COLMENIA and MiriadaX”. In Proc. IEEE International Symposium in Computers in Education, 169-172. 2014.
[Iniesto and Covadonga, 15] Iniesto, F., Covadonga, R.: “Accessible user profile modeling for academic services based on MOOCs”. In Proc. ACM International Conference on Human Computer Interaction. 2015.
[ISO, 12] ISO: “ISO 9241-171 Ergonomics of human-system interaction – Guidance on software accessibility”. 2012.
[Kalyvioti and Mikropoulos, 13] Kalyvioti, K., Mikropoulos, T.: “A virtual reality test for the identification of memory strengths of dyslexic students in higher education”. Journal of Universal Computer Science, 19(18), 2698-2721. 2013.
[Liyanagunawardena and Williams, 16] Liyanagunawardena, T., Williams, S.: “Elderly learners and massive open online courses: a review”. Interactive Journal of Medical Research, 5(1), 1- 11. 2016.
[Lohman, 15] Lohman, T.: “Mobile applications and web accessibility”. 2015. http://www.accessiq.org/news/features/2015/02/mobile-applications-and-web-accessibility
[NAD, 15] U.S. National Association of the Deaf: “NAD sues Harvard and MIT for discrimination in public online content”. 2015. http://nad.org/news/2015/2/nad-sues-harvard-and-mit-discrimination-public-online-content
[NZG, 13] New Zealand Government: “Web accessibility standard”. 2013. https://webtoolkit.govt.nz/standards/web-accessibility-standard-1-0/
[Martindale and Wiley, 04] Martindale, T., Wiley, D.: “Using weblogs in scholarship and teaching”. TechTrends, 49(2), 55-61. 2004.
[Molina, 10] Molina, R.: “Higher education for disabled students”. Research Journal, 34(70), 95-115. 2010.
[Morales, 07] Morales, A.: “White book on university and disability”. Grafo S.A Press. 2007.
[Nakata, 14] Nakata, K.: “WCAG 2.0 should not be applied to software and mobile apps”. Criptzone Insight. 2014. https://insight.cryptzone.com/accessibility/wcag-2-0-should-not-beapplied- to-software-and-mobile-apps/
[Notess and Lorenzen-Huber, 07] Notess, M., Lorenzen-Huber, L.: “Online learning for seniors: barriers and opportunities”. eLearn, 2007(5). http://doi.acm.org/10.1145/1266885.1266893
[OCR, 13] U.S. Department of Education Office of Civil Rights: “Resolution agreement South Carolina Technical College System”. 2013. http://www2.ed.gov/about/offices/list/ocr/docs/investigations/11116002-b.html
[Rodríguez-Ascaso and Boticario, 15] Rodríguez-Ascaso, A., Boticario, J. G.: “Accessibility and MOOCs: towards an integral perspective”. Ibero American Journal of Distance Education, 18(2), 61-85. 2015.
[Sanchez-Gordon and Luján-Mora, 13] Sanchez-Gordon, S., Luján-Mora, S.: “Web accessibility of MOOCs for elderly students”. In Proc. IEEE International Conference on Information Technology Based Higher Education and Training, 1-6. 2013.
[Sanchez-Gordon and Luján-Mora, 14a] Sanchez-Gordon, S., Luján-Mora, S.: “MOOCs gone wild”. In Proc. International Technology, Education and Development Conference, 1449-1458. 2014.
[Sanchez-Gordon and Luján-Mora, 14b] Sanchez-Gordon, S., Luján-Mora, S.: “Web accessibility requirements for massive open online courses”. In Proc. International Conference on Quality and Accessibility of Virtual Learning, 530-535. 2014.
[Sanchez-Gordon and Luján-Mora, 15a] Sanchez-Gordon, S., Luján-Mora, S.: “Adaptive content presentation extension for Open edX – enhancing MOOCs accessibility for users with disabilities”. In Proc. International Conference on Advances in Computer-Human Interactions, 181-183. 2015.
[Sanchez-Gordon et al., 15b] Sanchez-Gordon, S., Calle-Jimenez, T., Luján-Mora, S.: “Relevance of MOOCs for training of public sector employees: enrollment, completion and web accessibility challenges”. In Proc. IEEE International Conference on Information Technology Based Higher Education and Training, 1-5. 2015.
[Sanchez-Gordon and Luján-Mora, 15c] Sanchez-Gordon, S., Luján-Mora, S.: “An ecosystem for corporate training with accessible MOOCs and OERs”. In Proc. IEEE International Conference on MOOCs, Innovation and Technology in Education, 123-128. 2015.
[Sanchez-Gordon and Luján-Mora, 15d] Sanchez-Gordon, S., Luján-Mora, S.: “Accessible blended learning for non-native speakers using MOOCs”. In Proc. IEEE International Conference on Interactive Collaborative and Blended Learning, 19-24. 2015.
[Sanchez-Gordon and Luján-Mora, 16] Sanchez-Gordon, S., Luján-Mora, S: “Design, implementation and evaluation of MOOCs to improve inclusion of diverse learners”. In R. Mendoza-Gonzalez (Ed.), User-Centered Design Strategies for Massive Open Online Courses (MOOCs), 115-141. IGI Global. 2016.
[Shah, 14] Shah, D.: “MOOCs in 2014: breaking down the numbers”. edSurge. 2014. https://www.edsurge.com/news/2014-12-26-moocs-in-2014-breaking-down-the-numbers
[Shah, 15] Shah, D.: “MOOCs in 2015: breaking down the numbers”. edSurge. 2015. https://www.edsurge.com/news/2015-12-28-moocs-in-2015-breaking-down-the-numbers
[Singleton, 13] Singleton, K.: “Re-defining accessibility when it comes to MOOCs”. George Mason University. 2013. http://eportfolio.kjsingle.com/wp-content/uploads/2014/03/Accessibility-and- MOOCs_Final.pdf
[Sloan et al., 2010] Sloan, D., Atkinson, M., Machin, C., Li, K.: “The potential of adaptive interfaces as an accessibility aid for older web users”. In Proc. International Cross-Disciplinary Conference on Web Accessibility. 35-45. 2010.
[Smith, 12] Smith, M.: “Application of the ADA to websites: Congress should rely on the standards created by the World Wide Consortium”. 2012. http://works.bepress.com/marad_smith/1
[Stephanidis et al., 98]. Stephanidis, C. et al.: “Adaptable and adaptive user interfaces for disabled users in the AVANTI project”. In Proc. Intelligence in Services and Networks: Technology for Ubiquitous Telecom Services, 153-166. 1998.
[Tsai et al., 14] Tsai, H., Shillair, R., Cotten, S.: “Social support and playing around: an examination of how older adults acquire digital literacy with tablet computers”. In Proc. Research Conference on Communication, Information and Internet Policy. 130-161. 2014.
[United Nations, 08] United Nations: “Convention on the rights of persons with disabilities and optional protocol”. 2008.
[United Nations, 13] United Nations: “World population prospects”. 2013. http://esa.un.org/wpp/
[Vipond and Clarey, 15] Vipond, S., Clarey, J.: “Microsoft´s corporate MOOC”. 2015. [W3C, 99] W3C: “Weaving the web Berners Lee”. 1999. http://www.w3.org/People/Berners-Lee/Weaving/glossary.html
[W3C, 08] W3C: “Web content accessibility guidelines WCAG 2.0”. 2008. http://www.w3.org/TR/WCAG20/
[W3C, 09] W3C: “WAI early days”. 2009 http://www.w3.org/WAI/history
[W3C, 13] W3C: “Guidance on applying WCAG 2.0 to non-web information and communications technologies WCAG2ICT”. 2013. http://www.w3.org/TR/wcag2ict/
[W3C, 14a] W3C: “Web accessibility initiative”. 2014. http://www.w3.org/WAI/
[W3C, 14b] W3C: “Website Accessibility Conformance Evaluation Methodology WCAG-EM 1.0”. 2014. http://www.w3.org/TR/WCAG-EM/
[W3C, 15a] W3C: “Authoring tool accessibility guidelines ATAG 2.0”. 2015. http://www.w3.org/TR/ATAG20/
[W3C, 15b] W3C: “User agent accessibility guidelines UAAG 2.0”. 2015. http://www.w3.org/TR/UAAG20/
[W3C, 15c] W3C: “WAI guidelines and techniques”. 2015. http://www.w3.org/WAI/guid-tech.html
[W3C, 15d] W3C: “WAI guidelines and techniques for mobile accessibility”. 2015. http://www.w3.org/WAI/mobile/
[W3C, 15e] W3C: “Mobile accessibility: how WCAG 2.0 and other W3C/WAI guidelines apply to mobile”. 2015. http://www.w3.org/TR/mobile-accessibility-mapping/
[WHO, 11] World Health Organization: “Word Report on Disability”. 2011. http://www.who.int/disabilities/world_report/2011/en/
[Young, 14] Young H.: “Putting the U in MOOCs. The importance of usability in course design”. In Krause, S., and Lowe, C. (Eds.), Invasion of the MOOCs – the promises and perils of massive open online courses, 167-179. Parlor Press. 2014.