Challenges and Opportunities in Creating an Accessible Web Application for Learning Organic Chemistry

Allyson Grace Yu, Management Information Systems Department, Temple University, USA and Chemistry, Temple University, USA, allyson.yu@temple.edu

While students with disabilities demonstrate high interest in STEM during the transition from high school to college, their representation in STEM decreases throughout postsecondary education and into the workforce [13]. Organic chemistry, in particular, is a uniquely useful case study for exploring technological accessibility in STEM education due to its heavy reliance on highly visual components such as two- and three-dimensional representations of molecular structures. In addition, many university STEM programs recognize organic chemistry's rigorous perception as a “weed-out” course [4]. After a thorough search, no organic chemistry educational website has addressed level AA accessibility, and therefore, none have met the Web Content Accessibility Guidelines (WCAG 2.0). In this study, we investigated student's preferences on accessibility features while learning organic chemistry. We then explored “webORA,” a web-based application that allows the user to interact with a 3D molecular animation with subtitles that describe the reaction progression. We also evaluated the beta version of webORA by conducting user testing with users of multiple skill-levels. Lastly, we performed a manual accessibility audit of webORA.

CCS Concepts:Human Centered Computing → Web accessibility; • Usability testing → Redundancy; • Computer systems organization~Robotics;Networks~Network reliability;

Keywords: organic chemistry, device dependent event handlers, interactive, accessible technology

ACM Reference Format:
Allyson Grace Yu. 2022. Challenges and Opportunities in Creating an Accessible Web Application for Learning Organic Chemistry. In The 24th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '22), October 23–26, 2022, Athens, Greece. ACM, New York, NY, USA 6 Pages. https://doi.org/10.1145/3517428.3563284

1 PROBLEM AND MOTIVATION

In 2017, 19.8 percent of undergraduate students reported having a disability [11], including mobility and physical disabilities, emotional and mental health disabilities, vision and hearing disabilities, and health-related disabilities[14]. Despite the overall increase in students with disabilities enrolling in post secondary education, they are significantly underrepresented in STEM fields [12][11]. It is evident the lack of representation is not due to a lack of interest in STEM, as students with disabilities demonstrate high interest in STEM during the transition from high school to college. However, relatively few students with disabilities graduate with a STEM degree [9]. The reasons for this lack of representation is unclear, and ongoing studies are attempting to better understand why the lack of representation exists [12].

Though representation in STEM has been improving over the years, there is still a barrier for students with disabilities pursuing higher education in STEM fields due to societal and technological hurdles [6] [10]. With the rise of online learning in recent years and advancements in technology, undergraduate students must interact with many types of web pages or digital entry points such as: institutional sites, application portals, external web-based resources, etc. [13]. Researchers have found external web-based resources to be very beneficial in learning organic chemistry and other STEM subjects [3]. For example, a study based in Kigali city, Rwanda found organic chemistry students and professors supporting the rise of web-based learning (WBL) in organic chemistry classes [8]. However, problems arise when a web page or animation is not accessible to all targeted users.

Previous studies have shed light on the increasing use of technology and animations as effective instructional tools for students with disabilities [1]. Furthermore, organic chemistry is a STEM subject that has an important visual component because it relies heavily on two- and three-dimensional representations of molecular structures [5]. For example, a study on middle grade aged students with ADHD who use 3D, haptic-enabled VR for science learning demonstrates that 3D elements enhance the spatial awareness of students with ADHD [6].

Despite the advantages of 3D animations as an effective instructional tool for individuals with cognitive disabilities studying organic chemistry, there is no organic chemistry educational website that has been deemed fully accessible by the Web Content Accessibility Guidelines (WCAG 2.0). As discussed in previous studies, technology in the form of web-based resources, has the potential to mitigate this issue [6] [1] [5]. Therefore, this study focuses on the accessibility of external animation-based web resources for organic chemistry.

Many companies and organizations are starting to recognize the need for an accessible website. In 2020, more than 2,500 lawsuits were filed in federal court claiming websites were not designed to be accessible to people with disabilities, in violation of Title III [16]. Therefore, many web accessibility services such as accessibility overlays have risen in popularity. These modify the code of a web page with a snippet of JavaScript and usually come in the form of a toolbar, plugin, app, or widget, however, these do not make changes to the source code of the website [15]. These have been referred to as a “bandage fix” in which they act as an overlay to provide WCAG compliance. Therefore, they do not address the source code within the website. Some external STEM web-based resources, have developed accessibility checking of their own (Khan Academy), meanwhile other resources on the market do not address accessibility issues at all. However, unlike other resources currently on the market, we propose a newly developed site, web-based Organic Reaction Animations (webORA) which seeks to address accessibility from the start of development and thus implementation within the source code in order to improve understanding and reduce barriers to learning course material. This novel approach to addressing accessibility within the source code at the start is beneficial because it is scalable and allows modern developers to learn how to design for all in mind.

This paper has three distinct contributions. First, we collect questionnaire data gathered from 56 participants related to the development of an organic chemistry web application. Second, we record usability test data with 12 participants that focus on accessibility challenges that users may face when using this application. Finally, we present an accessibility analysis of a prototype web application that was created based on our previously collected data.

1.1 WebORA

Web-based Organic Reaction Animations (webORA) features 3D molecular reaction animations, allowing the user to interact with the reaction as it plays. Due to the rise of the importance of 3D interactive tools in the world of technology and learning, we addressed the question of how to make webORA more accessible? One goal of this project is to design webORA to address the gap in accessible online organic chemistry educational tools. In order to measure our success, we must also test its usability.

2 METHODS

First, a questionnaire was sent out to gather preliminary information from the target user group regarding their demographics and feelings towards web accessibility in STEM tools. Second, insights from this questionnaire were applied to designing the website prototype, in which a base template, also known as a bootstrap template, was used to create webORA [2]. Third, we tested the site using two methods: usability testing and manual accessibility testing. In order to find target user frustrations, we conducted usability tests with 12 participants. We quantified and solidified accessibility and usability issues by performing a manual accessibility audit of webORA. Since webORA is a work in progress we have been able to use the information from the questionnaires and testing to improve the teaching tool.

2.1 Questionnaire

2.1.1 Purpose of the Questionnaire. The aim of the questionnaire is to uncover information about the target user group. The questionnaire aimed to identify major problem areas within organic chemistry and accessibility of resources. Additionally, participants in the questionnaire are encouraged to sign up for the follow-up usability test.

  • Participants were asked about their background:
    1. Year in College:
    2. Gender identity:
    3. Racial/Ethnic identity:
    4. What is your major at Temple?
    5. Are you a first-generation college student?
    6. What organic chemistry course are you taking this spring 2022 semester?
    7. On a scale of 1-5: What is your understanding about web accessibility?
    8. In your own words, please describe what Web Accessibility means to you.
    9. To what extent do you agree or disagree with this statement: "My family supported my educational experience EMOTIONALLY throughout my life."
    10. To what extent do you agree or disagree with this statement: "My family supported my educational experience PHYSICALLY throughout my life." (i.e driving you to school or extracurricular activities)
    11. To what extent do you agree or disagree with this statement: "My family supported my educational experience FINANCIALLY throughout my life."
  • Participants were asked questions about their experiences with accessibility accommodations:
    1. Have you ever been diagnosed with a disability that impacts your learning experience. If you feel comfortable sharing, please list which specific disability (or disabilities).
    2. What accommodations, if any, do you typically need in STEM courses? (i.e. online software, subtitles, screen readers, extra time)
    3. To what extent do you agree or disagree with this statement: "In my previous education, I had access to disability resources prior to attending college."
  • Participants were asked questions about their perspective on accessibility features:
    1. To what extent do you agree or disagree with this statement: "SUBTITLES help me understand the content of a video or animation"
    2. To what extent do you agree or disagree with this statement, "While playing a reaction animation, having an AUDIO option to hear subtitles during the reaction would reduce a learning barrier for me."
  • Participants were asked if they would be interested in a follow-up interviews:
    1. Would you be interested in participating in a follow-up interview? Your feedback will help shape the development of the educational tool.
    2. If you selected yes in the previous question, we will reach out to you via email to schedule a 30 minute 1:1 interview. Please leave your email address below.

2.1.2 Participants. In total, 56 participants completed the questionnaire. Participants identified as freshmen, sophomore, juniors, and seniors. Four participants have only taken organic chemistry I and 52 participants have taken organic chemistry II. The gender identity of the participants was as follows: female = 37, male = 17, and non-binary = 2.

2.1.3 Questionnaire Study Method. A simple random sampling method was utilized to obtain responses from the questionnaire. The questionnaire was disseminated at a large public university during organic chemistry courses in the spring semester. Participants were notified their answers would not be shared with their instructor and would remain anonymous.

2.1.4 Summary of Questionnaire Results. 46 out of the 56 respondents said subtitles help them understand the content of a video or animation (agree or strongly agree). 18 out of the 56 indicated they would be interested in a follow-up usability test.

Figure 1
Figure 1: Screenshot of ©webORA organic reactions displayed

2.2 Development and Design of webORA

This portion of the project was in collaboration with researchers at Temple University (Steven A. Fleming) and Brigham Young University (Daniel H. Ess and Shusen Chen) who are supported on this project by the NSF Grants DUE #2120871 and #2121023, respectively. The website uses organic chemistry trajectories sourced from Jsmol, which is a web browser object that can be integrated into web pages. It is ideal for the development of web-based course materials and web-accessible chemical databases. The results from the questionnaire informed design decisions in the development of the beta version of webORA. A simple website layout was chosen to reduce distractions and increase accessibility of the site. Additionally, written text descriptions were added to each reaction page since 46 out of 56 participants indicated "subtitles" help them understand content of a video or animation.

2.2.1 Purpose of the Usability Test. Usability tests were conducted to gain feedback on the usability and intuitiveness of the webORA application for the target user.

2.2.2 Participants of Usability Test. In total, 12 participants were interviewed for usability testing and identified with studying in higher education in STEM. Participants were over the age of 18 years-old, with their year in college ranging from freshman to post-baccalaureate. The gender identity of the participants was as follows: female = 8, male = 2, and non-binary = 2. Participants reported on their level of experience studying organic chemistry as: has never taken an organic chemistry course (3), has taken organic chemistry one (1), has taken organic chemistry two (8). Additionally, two of the three participants that have not taken organic chemistry indicated that they will in the future for their majors. Out of the twelve students, ten self-identified with having a disability. P4, P9 and P11 identified with only having attention deficit hyperactivity disorder (ADHD). P3 has anxiety and obsessive compulsive disorder (OCD), P5 has ADHD, anxiety, and depression, P6 has ADHD and rheumatoid arthritis, P7 had a recent eye surgery that limits screen time on the computer, P8 has Ehlers-Danlos syndrome, P10 has convergence insufficiency, and P12 has ADHD, Bipolar; degenerative disc disease. P1 and P2 identified with not having a disability that impacts their learning experience. Participants reported spending at least 4 hours each day and at most 18 hours per day on the internet.

2.2.3 Usability Test Study Method. An email was sent to the previous questionnaire participants who indicated interest in a follow-up usability test as well as students registered with the university's disability resource center. The 30-minute virtual usability test was conducted through zoom in which the same researcher conducted the test utilizing a planned script. 1:1 usability test overview:

  • Participants were asked three preliminary questions
    1. Roughly how many hours a week altogether, would you say you spend using the Internet, including Web browsing and email, at work and at home?
    2. Do you have any favorite sites you prefer to interact with?
    3. How often do you use a 3D physical modeling kit?
  • Participants were asked to complete six tasks
    1. Task 1: What do you think this site is for? What are your first impressions? You may scroll just don't click anything.
    2. Task 2: Please locate the Sn2 reaction 1 on this website please identify what you think the buttons would do without clicking them.
    3. Task 3: Now play the reaction and comment on anything you notice. Does the 3D visual and selection of the transition state aid your learning?
    4. Task 4: Try to rotate the molecule in any way you please. End with the configuring that helps you the most with your learning. Did interacting with the molecule help with your spatial understanding of the reaction
    5. Task 5: Please locate the summarized written information about this reaction. Take a moment to read the text. Did reading this blurb of text facilitate your understanding of the Sn2 reaction animation you just played?
    6. Task 6: Please turn on the subtitles for the reaction (take your time). Please comment on the use of the dialogue box in terms of aiding your understanding of the reaction.
  • Participants were asked 3 follow-up questions
    1. Was the website intuitive and easy to use as a first time user?
    2. How likely are you to use a third-party organic chemistry website to aid your learning?
    3. Any additional comments on challenges you might have experienced learning stem subjects.
Figure 2
Figure 2: screenshot of Sn2 reaction page from ©webORA prototype

2.2.4 Summary of Usability Test Results. After completing the usability tests with the 12 participants, it was found that all but one participant uses external organic chemistry or STEM-related resources to aid their learning. This underscores the importance of this study for our target user group. For first-time users, webORA was described as having a “simplistic layout” (P11) that helps her focus on the information. Tasks 1 and 3 invoked the most incorrect responses and level of difficulty in completing the tasks for the participants. Task 1 asks to locate and play the SN2 reaction, and task 3 asks to rotate the molecule in the configuration that helps most with learning. It was found [1] playing the reaction and [2] zooming in and out using a trackpad caused frustrations for most participants. This accessibility issue requires an on click event to activate and can require quite a bit of coordination. Most found it “very cool you can rotate” the molecules around, especially while the reaction played (P6, P,P8, P10, P11). In terms of the website's ability to aid learning, participants commented it “helps with 3D spatial awareness” (P11) but it was hard to tell which finger motion does what. One participant, “blamed herself for being bad at technology” (P5) when trying to complete task 1 and 3. This participants reaction further eludes to how technology must be improved; it's not the user's fault. The questionnaire found subtitles helpful when learning. However, during task 4 and 5 of the usability test, though many students were still in support of subtitles on webORA, many voiced the benefits of the text being arranged in a more digestible format. For example, it would be “helpful to break down into bullet points and color code.”(P12). Overall, a student expressed the value in the text, saying it “would aid learning and would be “great to take notes down” (P9). Additionally, the text was easily readable when one participant accessed webORA via a dark mode reader.

2.3 Manual Accessibility Audit

2.3.1 Purpose of Manual Accessibility Audit. The Institute for Disability Research, Policy, and Practice recommends accessibility audits of early stage and developing websites [18]. Since webORA is in its early stages, a manual accessibility audit was completed in order to gain a baseline of accessibility issues to be addressed in the next phase of developing the website.

2.3.2 Manual Accessibility Audit Method. A template used to manually record accessibility issues was modified and sourced from an accessibility agency company. (see figure below). This study focused solely on testing the accessibility of one page of webORA: the Sn2 reaction page. The accessibility audit was tested against WCAG 2.0 and Level AA standards [7]

The web accessibility online tool called “Wave” by Webaim was used to run an audit of the accessibility issues to date of webORA. WAVE® is a suite of evaluation tools that helps authors make their web content more accessible to individuals with disabilities [17]. Based on researching the accessibility auditing services, the base requirements to deem a site accessible is:

  1. Comply with 30-40 percent of WCAG requirements (alt-text, color contrast, heading structure)
  2. Manual testing with real disabled intern users and assistive tech (screen readers and keyboard navigation - 60 -70 percent of WCAG requirements
  3. Maintain and training of accessibility practices

After compiling the audit, the study focused on the top two accessibility issues to focus on.

Table 1: Manual accessibility audit chart
Issue Name Description Severity Number of Observations User Impact
Device Dependent Event Handler Event handlers are triggered by a browser or user event, for example a user clicks the mouse. Device dependent event handlers are dependent upon use of a mouse (or touch) or keyboard. Other event handlers are device independent and are triggered by both the mouse/touch and keyboard, or by other means. Critical 44 Someone who only navigates through keyboard will not be able to use this feature
Missing Form Labels If a form control does not have a properly associated text label, the function or purpose of that form control may not be presented to screen reader users. Form labels also provide visible descriptions and larger clickable targets for form controls. High 110 If there are no form labels with buttons right next to each other it may be easy to mix up the functions and cause frustration for screen reader users.

2.3.3 Manual Accessibility Audit Results. Based on the accessibility report, device dependent handlers were found to create one of the biggest barriers for many of the participants. The accessibility audit shows 44 instances which underscores this issue. This accessibility issue may be tied to the frustrations expressed when students had a hard time rotating the molecule in task 4 of the usability test. Based on the accessibility audit,the reaction page requires an on-click event to start moving the molecule around. Users who navigate the web through only a keyboard or a mouse will face barriers to access of this feature. Additionally, there were missing form labels which would cause a barrier for a user who may have low to no vision and requires a screen reader to read aloud the functions of a button on a screen. The accessibility audit showed 110 instances. This number is likely much higher in the audit due to having more form labels per reaction. (See table below for more details) When testing webORA for keyboard navigation, most buttons were able to be clicked or interacted with on the screen when putting a Windows computer in screen-reader mode. For example, the space bar was able to play and pause the reaction. One category where webORA performed well against the accessibility audit and had minimal issues detected was in its color contrast. This is also supported by the usability tests in which one participant used the site with a dark-mode reader and was able to interact and complete the tasks with webORA with no contrast issues.

3 DISCUSSION

According to the follow-up questions after user testing, webORA was deemed usable and intuitive for most first-time users. Additionally, many students asked the researcher if they would be allowed to save and use the site, indicating that the use and function appeal to its audience. However, the site did not fully pass the WCAG accessibility guidelines. One of the most notable accessibility findings was that device dependent-event handlers were a common accessibility issue. With the increase in 3D academic modeling tools and societal recognition of augmented reality technology, addressing the use of device-dependent handlers will be essential to making sure this evolution in technology does not leave anyone out. In conclusion, due to the rise in interactive external STEM resources, device-dependent event handlers have great potential for use in accessible virtual STEM tools. Developers must keep in mind that, while these technologies have great potential to aid in learning for those with disabilities, inadequate accessibility tools, such as inadequate device-dependent event handlers, as shown with webORA, may create barriers to access or frustrations in its usability for students. Expansion of STEM resource accessibility in education, and thus expansion of the tech accessibility in general, are key to including communities that are often underrepresented or feel unseen in STEM fields.

4 LIMITATIONS AND FUTURE DIRECTIONS

Due to time limitations, only the WAVE evaluation tool was used during the manual accessibility auditing, therefore, the manual accessibility audit is subject to minor inaccuracies. Additionally, the sample size for the usability testing was small (12 participants) and did not encompass all disabilities. Due to resource constraints, this study solely tests the Sn2 reaction page for usability tests. Since webORA was created as a single page website, and the WAVE evaluation tool has a constraint of accessing one page at a time, the results of the audit had caught errors from the entire webORA website. Therefore, results were higher than expected and the audit results had to be explored further to find specific issues reported on the Sn2 page. Thus, only two main issues were reported on.

For future studies, it would be beneficial to test webORA against other accessibility evaluation tools. Additionally, more research should be conducted on the use of device dependent event handlers in other interactive chemistry educational websites. Lastly, further studies should look into device dependent event handlers in the scope of other STEM subjects.

ACKNOWLEDGMENTS

I would like to thank the broader team that contributed to the larger project supported by NSF (#2120871): Steven A. Fleming, Daniel Ess, and Shusen Chen. I am grateful to our participants. I also appreciate feedback and support from fellow mentors Will Bubenik, Munir Mandviwalla, Long Nguyen, Kit Anne Aronoff. Lastly, I want to appreciate thanks to my accessibility mentor Michael Crabb, fellow students (Emily Vergara and Josh Schneider), and anonymous reviewers.

REFERENCES

  • Basak Baglama, Yucehan Yucesoy, and Ahmet Yikmis. 2018. Using animation as a means of enhancing learning of individuals with special needs. TEM Journal 7, 3 (2018), 670.
  • Start Bootstrap. 2021. Bootstrap Templates & Themes. (2021). https://startbootstrap.com/themes
  • YunJeong Chang and Seung Won Park. 2014. Exploring Students’ Perspectives of College STEM: An Analysis of Course Rating Websites.International Journal of Teaching and Learning in Higher Education 26, 1(2014), 90–101.
  • Michael T Crimmins and Brooke Midkiff. 2017. High structure active learning pedagogy for the teaching of organic chemistry: Assessing the impact on academic outcomes. Journal of Chemical Education 94, 4 (2017), 429–438.
  • Gabriela A Fernández, Romina A Ocampo, Andrea R Costantino, and Néstor S Dop. 2019. Application of didactic strategies as multisensory teaching tools in organic chemistry practices for students with visual disabilities. Journal of Chemical Education 96, 4 (2019), 691–696.
  • Rebecca Hite, Gina Childers, Gail Jones, Elysa Corin, and Mariana Pereyra. 2020. Describing the experiences of students with ADHD learning science content with emerging technologies. Journal of Science Education for Students with Disabilities 24, 1(2020), 12.
  • Web Accessibility Initiative. 2020. Web Content Accessibility Guidelines (WCAG) 2 Level AA Conformance. (2020). https://www.w3.org/WAI/WCAG2AA-Conformance
  • Aloys Iyamuremye, Janvier Mukiza, Ezechiel Nsabayezu, Fidele Ukobizaba, and Kizito Ndihokubwayo. 2022. Web-based discussions in teaching and learning: Secondary school teachers’ and students’ perception and potentiality to enhance students’ performance in organic chemistry. Education and Information Technologies 27, 2 (2022), 2695–2715.
  • Ahlam Lee. 2014. Students with Disabilities Choosing Science Technology Engineering and Math (STEM) Majors in Postsecondary Institutions.Journal of Postsecondary Education and Disability 27, 3(2014), 261–272.
  • Dorothy L. Miner, Ron Nieman, Anne B. Swanson, and Michael Woods. 2001. Teaching Chemistry to Students with Disabilities: A Manual for High Schools, Colleges, and Graduate Programs. (2001). https://www.acs.org/content/dam/acsorg/education/publications/teaching-chemistry-to-students-with-disabilities.pdf
  • NCES. 2021. National Center for Science and Engineering Statistics (NCSES) Women, minorities, and persons with disabilities in science and engineering. Social,Behavioral and Economic Sciences(2021).
  • Mariel A Pfeifer, Eve Melanie Reiter, Julio J Cordero, and Julie Dangremond Stanton. 2021. Inside and out: Factors that support and hinder the self-advocacy of undergraduates with ADHD and/or specific learning disabilities in STEM. CBE—Life Sciences Education 20, 2 (2021), ar17.
  • Erin Scanlon, Zachary W Taylor, John Raible, Jacob Bates, and Jacquelyn J Chini. 2021. Physics webpages create barriers to participation for people with disabilities: five common web accessibility errors and possible solutions. International Journal of STEM Education 8, 1 (2021), 1–16.
  • George A Scott. 2010. Higher education and disability: Education needs a coordinated approach to improve its assistance to schools in supporting students. DIANE Publishing.
  • The A11Y Project Project Team. 2021. Should I use an accessibility overlay. (2021). https://www.a11yproject.com/posts/should-i-use-an-accessibility-overlay/
  • Minh Vu, Kristina Launey, and Susan Ryan. 2022. ADA Title III Federal Lawsuit Filings Hit An All Time High. (2022). https://www.adatitleiii.com/2022/02/ada-title-iii-federal-lawsuit-filings-hit-an-all-time-high/
  • Webaim. 2022. WAVE Web Accessibility Evaluation Tool. (2022). https://wave.webaim.org/
  • Jon Whiting. 2012. Four keys to system-wide web accessibility.(2012). https://webaim.org/blog/four-keys-to-accessibility/

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org.

ASSETS '22, October 23–26, 2022, Athens, Greece

© 2022 Association for Computing Machinery.
ACM ISBN 978-1-4503-9258-7/22/10…$15.00.
DOI: https://doi.org/10.1145/3517428.3563284