Open Educational Resources (OER)

Evaluating findability of open educational resources from the perspective of users with disabilities: a preliminary approach

Rosa Navarrete, Sergio Luján-Mora. 2015 Second International Conference on eDemocracy & eGovernment (ICEDEG 2015), p. 112-119, Quito (Ecuador), April 8-10 2015. ISBN: 978-3-9075-8910-6.

Abstract

e-Education is an umbrella term that refers to educational collaborative environments supported by technology that related not only formal academic institutions programs but also self-learners and teachers around the world. This educational paradigm is of paramount importance for society nowadays, because expands access opportunities to education at all levels. These learning environments can take advantage of Open Educational Resources. Using OER is a growing trend and it is turning into a recommendation by some governments and international education forums. Nevertheless, in spite of the availability of OER websites, quite a few of them do not consider accessibility design principles, therefore these websites present access barriers for users with disabilities. Besides, only a reduce number of these websites offer accessible resources, i.e. digital contents suitable for users with disabilities. On the other hand, in web environment, findability is a design consideration that improves the user's searchable experience by simplifying the location and acquisition of elements in websites. In this restricted scenario, findability of educational resources is a key issue for users with disabilities. In this research, we address the issue of findability of Open Educational Resources (OER), from the perspective of inclusive participation of users with disabilities, by considering three aspects: web accessibility, some usability best practices oriented to users with disabilities and Information Architecture. The evaluation is performed in some important and well known OER websites.

Keywords: Open Educational Resources, findability, web accessibility, usability, Information Architecture, e-Education

More information: Evaluating findability of open educational resources from the perspective of users with disabilities: a preliminary approach

Download: Evaluating findability of open educational resources from the perspective of users with disabilities: a preliminary approach

  1. I. INTRODUCTION
  2. II. E-EDUCATION AND OPEN EDUCATIONAL RESOURCES
  3. III. FINDABILITY ON WEBSITE
  4. IV. EVALUATION METHOD
  5. V. RESULTS AND DISCUSSION
  6. VI. CONCLUSION AND FUTURE WORK
  7. ACKNOWLEDGMENT
  8. REFERENCES

I. INTRODUCTION

Guarantee the access to education is one of the biggest challenges for current society. It implies that people regardless to their physical, neurological, and environmental limitations, have the opportunity to improve their educational conditions.

e-Education is based on learning environments, supported by technology, that enable a collaborative learning experience. Open Educational Resources (OER) can be used in such environments.

The use of OER has been recognized by UNESCO in “2012 Paris OER Declaration”, as a strategy to improve access to quality education [1].

Users with disabilities can use OER but under accessibility condition, that is, the website must meet accessibility principles, and the resources must be accessible on depending their formats.

Nowadays, despite the growing online availability of OER, not all the OER websites are accessible, so they are not able to serve in equality conditions to all learners including people with disabilities. Besides, only few websites offer OER which are accessible.

In this restricted scenario in relation to availability of OER findability of resources is a certainly important issue. In web environment findability is a design consideration that improves the user's searchable experience by simplifying the location and acquisition of elements on the websites [2].

A failed search of accessible resources, not only implies unsatisfied users but also valuable educational resources that cannot be used. Findability of resources that are more relevant to users’ particular needs is indeed a different kind of obstacle that disabled learners must deal.

Findability is one of the aspects considered by Information Architecture (IA) in web design. IA embraces the organization, navigation and search into the website, which determine how the users accomplish tasks, how they understand the content distribution on the website, and also how they find what they need.

In this research we evaluate findability of resources in some of the most important OER websites, with the foreknowledge that these offer accessible resources. This evaluation verifies accessibility, usability, and IA recommendations.

II. E-EDUCATION AND OPEN EDUCATIONAL RESOURCES

e-Education refers to education collaborative environments related not only to formal academic institutions programs but also self-learners and teachers around the world. These teaching-learning platforms preferably on web environments are based on an effective combinations of pedagogy and technology [3]. Besides, e-Education contributes to expand opportunities to quality learning and to redress inequalities in access to education [4]. To aim this purpose, e-Education environments can take advantage of OER.

OER are teaching, learning and research resources that resides in repositories available through a website, that have been released under intellectual property license that permits their free use or re-purposing by others [5]. OER include a wide variety of digital pieces in distinct formats: full courses in web format, course materials in media format, interactive elements, textbooks, streaming videos, evaluation tests, simulation software, and many other kind of resources to support knowledge.

The importance of OER in the democratization of access to knowledge and to educational opportunities across the globe has been formally declared by UNESCO in the 2012 World Open Educational Resource Congress [1], and also promoted by government instances like the European Commission [6], and United States of America [7].

In the developing countries, where many students are not able to afford quality education in university degrees, OER can provide them to access to educational material from outstanding universities, including entire courses of degree programs [8] [9].

According to the European Commission [10], the current trend from an ageing population demands a new approach to education and training within the framework of "Lifelong Learning". This term embraces all learning activities undertaken throughout life, with the aim of improving knowledge, skills and competence. The use of OER for self-learning can contribute with Lifelong Learning.

A. Disabilities and Open Educational Resources

World Health Organization (WHO) [11] affirms that over 1 billion people, 15% of the global population live with some form of disability. Besides, elderly world's population will increase from 600 million to 2 billion until 2050 [12], which means that there will be more people with disabilities consequence of aging.

In relation to web use, disability is a term that embraces mobility impairments (restricted movement of arms, hands and fingers), sensory impairments (visual or hearing), and cognitive impairments (thinking, memory, language, learning and perception). Besides, it includes impairments inherent to aging conditions, and participation restrictions due to technological limitations [13].

Convention on the Rights of Persons with Disabilities [14] promotes equal enjoyment of all human rights by people with disabilities, it includes education as a fundamental right. To achieving this goal is precise that learning environment be inclusive, that is, website and educational resources themselves should be accessible to ensure access for all people regardless their abilities and disabilities.

B. Open Educational Resources Websites

OER are available through websites. At this time, it is not possible to give an accurate estimation of the number of OER websites, because there is an increasing trend in universities and other educational communities, in a global scale, to join to this initiative.

This configures a dynamic scenario for creating websites, but not for their sustainability. Some surveys have detected the creation and disappearing of the websites [15] [16]. Because of this, there are no formal directories of OER websites, although we can find some references [17] [18]. Many OER websites have small scale activities, while some others are well known and rated in the academic ambience.

The volatility of OER websites is also important for findability of resources, because users need to be informed about the websites where they can look for resources.

III. FINDABILITY ON WEBSITE

Findability, refers to find content or functionality that user assume is present in a website, according to its purpose [2]. So it is closely related to the quality of response that users obtain when they define a search to locate what they are looking for.

On OER websites, high findability is a desirable condition, because the primary function of the website is to provide users the content that they required. The ability to find educational resources suitable for specific requirements of users with disabilities, will increase the overall credibility and utility of the website.

In this research we evaluate findability of open and accessible educational resources on websites considering these parameters:

A. Web accessibility and web usability

The definition and scope of the terms, web accessibility and web usability have been widely discuss in literature trying to define the prevalence of one of them [19] [20]. Although usability and accessibility overlap in many aspects, the achievement of one of them doesn’t imply the achievement of the other one, and vice versa. This condition was clearly stated by Joe Clark, web accessibility expert, in the following quote: “Usable sites can be inaccessible. Conversely, accessible sites can be unusable” [21].

Nevertheless, there is a consensus about that web accessibility and web usability are closely related, and both should be considered to improve the user experience, and address the needs of people with disabilities [22].

1) Web accessibility

The goal of web accessibility is to ensure that people with disabilities can use the web in equal conditions than others. Accessibility on the websites and in the educational resources is the major requirement to guarantee equal right of accessing for all users, especially for disabled people.

An important standard that establishes the principles for accessibility is WCAG 2.0. This is a technical standard of 2008 [23], released by World Wide Web Consortium (W3C), Web Accessibility Initiative (WAI), and was approved as an ISO/IEC 40500 International accessibility standard in October 2012.

WCAG 2.0, has 12 guidelines organized under 4 principles: Perceivable (users must be able to perceive the information being presented), Operable (users must be able to operate the interface), Understandable (users must be able to understand the information as well as the operation of the user interface), and Robust (users must be able to access the content as technologies advance). It is based on a set of success criteria that are technology neutral. Conformance to the WCAG is defined on ordinal levels (A, AA, and AAA) from obligatory, recommended and desirable.

The evaluating of web accessibility can be carried out with automatic evaluation tools for a preliminary analysis. A formal validation should be complement with a human expert to verify and interpret the results.

Automated tools have distinct coverage, completeness and correctness with respect to WCAG 2.0. Whereby, according to a recent study [13], it is recommendable to use a combination of some automated tools in web accessibility analysis in order to achieve reliable evaluation.

2) Web usability

The standard ISO 9241-11: Guidance on Usability (1998), define usability as "the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use".

Web usability refers to the quality of a user's experience when interacting with websites, based on effective completion of their tasks on the website [24] [25]. Web usability contributes to better experiences of people with disabilities and without disabilities in web navigation [22].

Web usability defines some characteristics that should have a website [25] [26]:

Some characteristics like fault tolerance, are associated to web applications more than to websites.

Web usability evaluation aims to recognize explicit usability problems in website and has some common elements with web accessibility evaluation [19].

There are distinct approaches to classify usability evaluation methods according to expert opinions [28], but, the discussion and comparison of these are out of the interest of this work.

For the purpose of this research we decide to use an empirical review based on the perspective of users with disabilities and findability of resources.

B. Information Architecture

IA embraces structure and organization of a website, including labeling, search, and navigation systems [29]. These components are described below:

IA has some common aspects with web accessibility and web usability such as: content structure that improve comprehension, navigation system through menus that guides to user in exploration of website, labeling meaningful to user, a correct hierarchy of label headings that enables converting the content into a linear presentation for screen reader that is intelligible to blind users.

For this research we evaluate the contribution of IA to findability of educational resources considering the aspects mentioned above.

IV. EVALUATION METHOD

The primary goal of this research is to evaluate findability in the OER websites, paying attention to the special needs of users with disabilities. For the evaluation we consider: web accessibility, web usability and IA (structure, navigation and searching).

Although the evaluation is conducted in a separate way for each parameter, it is important to emphasize that an unsatisfactory evaluation result of one of them, implies a low expectation about success of the findability.

A. Websites for evaluation

We select some websites for evaluation based on their permanency and recognizing in academic ambience and the foreknowledge that these websites offer accessible educational resources [30]. The evaluation of distinct parameters that affect findability of resources in these websites will be used as an approximation of the current situation.

We present the evaluation on three websites:

B. Web accessibility evaluation

If users with disabilities encounter barriers for access to home page on the website, there is no way that they can navigate and find the resources. Accordingly in this research, we evaluate the accessibility in the home page, to verify access problems, and to get a reference about accessibility conformance level of the entire website.

In this research the evaluation of web accessibility is conducted through the application of three automated tools and it is complemented with evaluation of HTML and CSS in the web page. The automated tools are:

As another criteria of quality of web page, we validate HTML and CSS, using W3C Validation Service [34] [35].

C. Web usability evaluation

The evaluation of web usability is conducted from the perspective of that users with disabilities can successfully complete the searching and retrieve of educational resources, considering their particular requirements about navigation alternatives, the clarity of contents and the effort required to operate the forms.

We evaluate usability through an empirical approach, considering a set of 247 guidelines used by Userfocus [36], a company related to usability improving in websites. These guidelines are extracted from international standard, ISO 9241-151:2008 that provides guidance on the human-centered design of web user interfaces for increase usability.

The evaluation based on this set of guidelines, allows inspection and respective valuation in a checklist, with rating 1 (doesn't comply with the guideline), 1 (complies with the guideline) or 0 (needs improve compliance). If the guideline is not relevant to the evaluation purpose, the rating is left blank.

The final score obtained shows website's usability level. However, it is necessary to note that all of the criteria have been weighted equal, so it is pertinent consider the relevance of each of the guidelines for the purpose of this research. For example, complying with the guideline: “The home page contains a search input box”, it is more important for findability of resources, than complying with the guideline, “The site is pleasant to look at”.

The guidelines considered for this research are those relevant to educational resources websites, and are reviewed from the perspective of disabled users with emphasis in findability of educational resources.

D. Information Architecture evaluation

The evaluation of IA is performed by reviewing empirical of these components:

V. RESULTS AND DISCUSSION

Due to the lack of space, it is not possible to include full details of each analysis, hence we present a helpful summary for each evaluation.

A. Results of web accessibility evaluation

Table I, shows the most important results about accessibility evaluation through automated tools.

The score reported by eXaminator is considered as an accessibility measure for the purpose of this research. MERLOT has the best score in eXaminator, but it is still low. This value indicates that the home pages of websites have access barriers for users with disabilities, particularly for blind users and those with limited mobility of upper limbs.

The highest value reported as known problems by Achecker for OER Commons represents basically multiple occurrences of insufficient contrast ratio between link text and background, for a specific region in the page, so can be discarded as a problem.

It is important to recognize that all websites have minimal errors in reviewing HTML and CSS. It means that these websites meet W3C standards.

TABLE I. RESULTS OF ACCESIBILITY EVALUATION

B. Results of web usability evaluation

Table II shows results of manual evaluation based on adaptation of Userfocus from ISO 9241-151:2008 guidelines. Column labelled “Guid.”, shows the number of guidelines in that subject, column labelled “Answers”, corresponds to the number of guidelines considered for this evaluation, column “RS” shows the Raw Score obtained for evaluation of the respective guidelines for each websites, an column “Score” is calculated as percentage to the maximum score possible for each guideline, as shown below:

Score (%) = (RS + Answers) / (2* Answers)

The results shows a good overall score over 80% for all the websites with exception of OCW UPM with 63%.

For the purpose of this research the guidelines concerning to Home Page, Task Orientation, Navigation and IA, Forms and Data Entry, and Search with values below 60% represent a barrier for users with disabilities. It is because these aspects influence in the efficiency and preciseness of navigating and searching.

From the perspective of users with disabilities, guidelines related to Trust and Credibility, Page Layout and Visual Design, Writing and Content Quality are not significantly relevant to effective use of OER websites.

Although usability issues are not critical in relation with access to a website for users with disabilities, however these hinder the findability of educational resources. A website designed according to best usability practices allows more productive experience when users explore the website.

TABLE II. RESULTS OF USABILITY EVALUATION ISO 9241-151:2008

C. Results of Information Architecture evaluation

Table III shows a summary of reviewing of IA evaluation parameters: structure, navigate, and search. The "Structure" column includes the interface aspect and content structure (how the content is grouped so that be intelligible and straightforward for users). The "Navigation" column includes annotations about the menus available on website: main menu, breadcrumbs (so the user knows where he is and where he comes from), link to Home page (so users can take new actions). The “Search” column has annotations about distinct ways to search: “Basic” by a keyword in a search box, “Advanced” for refinement of searching with some parameters, and “Browse” for navigate in a grouped list of resources.

In literature review on this subject we don't find a metric for IA evaluation, therefore, in order to assign a weight to information concepts, we use a qualification in a scale from 1 to 3 for each of the elements evaluated for IA, according to annotations in Table III. The value 1 is assigned as minimum, the value 2 when the element exists but has some issues, and the value 3 when element is highly regarded in that website.

TABLE III. SUMMARY OF IA EVALUATION

The values presented in Table IV shows the score assigned for each parameter on a total of 18 points. The abbreviations in column headings represent: "I" for Interface, "CO" for Content Organization, "Bas." for Basic, "Adv." for Advanced, “Brow." for Browse.

D. Combined evaluation of all parameters

As far as we know, there is not a metric for weighing the combined importance of web accessibility, web usability and IA, in web design; for that reason, we present a qualitative evaluation about the combined evaluation results. For this, we consider the score of eXaminator tool that appears in Table I for accessibility qualification, the percentage value of usability obtained in Table II, and the percentage value of IA in Table IV. We see results in Table V. In all websites there are accessibility issues that hinder access for people with disabilities.

However, the results show that OER Commons and MERLOT have better design to serve users with disabilities and hence both provide success in findability of resources. On the other hand, OCW UPM has issues for accessibility, usability and IA that are obstacles to success in findability of resources.

It is important to annotate that OER Commons presents multiple problems of insufficient contrast ratio between link text and background, for a specific region in the page. This condition produces an error notification for the automated tools. For that reason, eXaminator tool assigns a lower accessibility score to OER Commons. However, this is not a real problem, because the region in page where the problem is detected is not relevant. Additionally, OER Commons includes "Learner Options" menu that allows to adjust viewing preferences which is particularly helpful for users with disabilities.

TABLE IV. PONDERATIONS OF IA EVALUATION

Furthermore, navigation on OER Commons website is task-oriented, easy to follow intuitively and friendly, and it makes possible to explore the categories and subcategories of resources. This represents an advantage to users with disabilities.

TABLE V. RESULTS OF COMBINED EVALUATION

VI. CONCLUSION AND FUTURE WORK

This research is a preliminary approach to findability of OER in websites, so although it is not possible to make conclusive claims, it is feasible to present some appreciations.

Use of OER is an effective strategy for increase educational opportunities for all people, but its potential is still out of reach for users with disabilities. Even websites that are theoretically accessible have accessibility and usability issues for people with disabilities.

In this research we evaluate some prominent OER websites and we found that from the perspective of people with disabilities, none of them guarantee access without barriers and hence findability of resources is unsatisfactory. It is necessary a deeper evaluation process and user testing to define specific problems for each disability.

However, web usability has better application in the websites, although OCW UPM needs to improve in this aspect. IA recommendations also needs to be addressed in OCW UPM.

MERLOT and OER Commons, have best evaluations results for three parameters. These could be suitable for people with disabilities with better perspective for findability success.

In the context of the IA, we observed that each of the websites reviewed sets its own content categorization and labelling. This is confusing for users with disabilities, for example, blind people or people with visual impairments that use assistive technology are not able to find similarities when they navigate on each website.

It is necessary to encourage to all people related to OER initiatives (designers of websites and resources repositories as well as academics that release educational resources) to raise awareness about developing OER websites suitable for all users regardless their disabilities.

We plan to complement this type of analysis in OER that offer content in Spanish language and according to the results, we can recommend some of them for people with disabilities in Spanish speaking countries.

Finally, in a near future, we want to address a user centered design applicable to OER websites that includes usability and accessibility considerations.

ACKNOWLEDGMENT

This work has been partially supported by the Prometeo Project by the Secretary of Higher Education, Science Technology and Innovation (SENESCYT) of the Ecuadorian Government.

REFERENCES

[1] UNESCO, UNESCO, Junio 2012. [Online]. Available: http://www.unesco.org/new/fileadmin/MULTIMEDIA/HQ/CI/CI/pdf/Events/Paris%20OER%20Declaration_01.pdf.

[2] J. Cardello, "Nielsen Norman Group," Low Findability and Discoverability: Four testing methods to identify the causes. [Online]. Available: http://www.nngroup.com/articles/navigation-ia-tests/.

[3] J. M. Broekman and M. H. Foox, e-Education and the Web, iUniverse, 2006.

[4] "White Paper on e-Education," Government Gazette, no. 26734, pp. 14-16, 2006.

[5] D. Atkins, J. Seely Brown and A. L. Hammond , "A review of the Open Educational Resources (OER) Movement: Achivements, Challenges and New Opportunities," February 2007. [Online]. Available: http://www.hewlett.org/uploads/files/ReviewoftheOERMovement.pdf.

[6] European Commission, "European Commission Press Release Database," 11 2012. [Online]. Available: http://europa.eu/rapid/press-release_IP-12-1233_en.htm.

[7] The Open Government Partnership, "Announcing New Open Government Initiatives as part of the Second Open Government National Action Plan for the United States of America," September 2014. [Online]. Available: http://www.whitehouse.gov/sites/default/files/microsites/ostp/new_nap_commitments_report_092314.pdf.

[8] C. Hodgkinson-Williams and P. Arinto, "Open Education for a Multicultural World: A report from the Research on Open Educational Resources for Development Project in the Global South," 04 2014. [Online].

[9] UNESCO, Commonwealth of Learning, "Guidelines for Open Educational Resources (OER) in Higher Education," 2011.

[10] "European Commission, Education and Training," 2001. [Online]. Available: http://ec.europa.eu/transparency/regdoc/rep/1/2001/EN/1-2001-678-EN-F1-1.Pdf.

[11] World Health Organization, "Disabilities and rehabilitation," [Online]. Available: http://www.who.int/disabilities/actionplan/en/.

[12] World Health Organization, "Ageing and Life Course," September 2014. [Online]. Available: http://www.who.int/ageing/about/facts/en/.

[13] M. Vigo, J. Brown and V. Conway, "Benchmarking web accessibility evaluation tools: measuring the harm of sole reliance on automated tests," in Proceedings of the 10th International Cross-Disciplinary Conference on Web Accessibility, ACM, New York, USA, 2013.

[14] United Nations, "Conventions on the Rights of Persons with Disabilities and Optional Protocol," 2006. [Online]. Available: http://www.un.org/disabilities/documents/convention/convoptprot-e.pdf.

[15] A. Tzikopoulos, N. Manouselis and R. Vourikari, "An Overview of Learning Objects Repositories," in Learning Objects for Instruction, Idea Group Publishing, 2007, pp. 29-55.

[16] F. Neven and E. Duval, "Reusable Learning Objects: A survey of LOM-Based Repositories," in Proceedings ACM Conference Multimedia, 2002.

[17] Discover and Open Source World, "A guide to free and open source education," [Online]. Available: http://opensource.com/education/13/4/guide-open-source-education.

[18] OER Quality Project, "Directory of OER repositories," [Online]. Available: http://oerqualityproject.wordpress.com/2012/10/22/directory-of-oer-repositories/.

[19] H. Petrie and O. Kheir, "The Relationship Between Accessibility and Usability of Websites," in SIGCHI Conference of Human Factors in Computing Systems, San José, California - USA, 2007.

[20] A. L. Dias, R. Pontin de Mattos Fortes and P. C. Masiero, "Increasing the Quality of Web sytems: By inserting requirements of accessibility and usability," in Eight International Conference on the Quality of Informations and Communications Technology (QUATIC), 2012.

[21] J. Clark, Building accesible websites, New Riders, 2002.

[22] W3C, "Web accessibility and Usability working together," [Online]. Available: http://www.w3.org/WAI/intro/usable.

[23] W3C-WAI, "Web Content Accessiblity Guidelines (WCAG) 2.0," [Online]. Available: http://www.w3.org/TR/WCAG20/.

[24] Usability.gov, "Usability Evaluation Basics," [Online]. Available: http://www.usability.gov/what-and-why/usability-evaluation.html.

[25] J. Nielsen, "Nielsen & Norman Group: Usability 101, Introduction to usability," [Online]. Available: http://www.nngroup.com/articles/usability-101-introduction-to-usability/.

[26] Usability.gov, "What & Why of Usability," [Online]. Available: http://www.usability.gov/what-and-why/usability-evaluation.html.

[27] D. A. Norman, "Affordance, Conventions and Design," Magazine Interactions, vol. 6, no. 3, pp. 38-43, May/June 1999.

[28] M. Aamir y A. Mansoor, «Testing Web Application from usability perspective,» de Computer,Control & Communication (IC4), 2013 3rd International Conference, 2013.

[29] P. Morville and L. Rosenfeld, Information Architecture for the World Wide Web, 3rd Edition ed., O'Reilly, 2006, p. 4.

[30] R. Navarrete and S. Luján-Mora, "Accessibility considerations in Learning Objects and Open Educational Resources," in ICERI2013 Proceedings, Sevilla - Spain, 2013.

[31] AChecker, "Web Accessibility Checker," [Online]. Available: http://achecker.ca/checker/index.php.

[32] eXaminator, "Evaluación de la accesibilidad web," [Online]. Available: http://examinator.ws/.

[33] TAW, "TAW3 online," Fundación CTIC, [Online]. Available: http://www.tawdis.net/ingles.html?lang=en. [Accessed 10 10 2014].

[34] W3C, "Markup Validation Service," [Online]. Available: http://validator.w3.org/.

[35] W3C, "CSS Validation Service," [Online]. Available: http://jigsaw.w3.org/css-validator/validator.html.en.

[36] Userfocus, "Usability consultancy," [Online]. Available: http://www.userfocus.co.uk/consultancy/index.html.

More information: Evaluating findability of open educational resources from the perspective of users with disabilities: a preliminary approach

Download: Evaluating findability of open educational resources from the perspective of users with disabilities: a preliminary approach

← Back to Publications