A toolkit for successful workplace learning analytics at software vendors
- Authors: Whale, Alyssa Morgan
- Date: 2024-04
- Subjects: Computer-assisted instruction , Intelligent tutoring systems , Information visualisation
- Language: English
- Type: Doctoral theses , text
- Identifier: http://hdl.handle.net/10948/64448 , vital:73713
- Description: Software vendors commonly provide digital software training to their stakeholders and therefore are faced with the problem of an influx of data collected from these training/learning initiatives. Every second of every day, data is being collected based on online learning activities and learner behaviour. Thus, online platforms are struggling to cope with the volumes of data that are collected, and companies are finding it difficult to analyse and manage this data in a way that can be beneficial to all stakeholders. The majority of studies investigating learning analytics have been conducted in educational settings. This research aimed to develop and evaluate a toolkit that can be used for successful Workplace Learning Analytics (WLA) at software vendors. The study followed the Design Science Research (DSR) methodology, which was applied in iterative cycles where various components of the toolkit were designed, developed, and evaluated by participants. The real-world-context was a software vendor, ERPCo, which has been struggling to implement WLA successfully with their current Learning Experience Platform (LXP), as well as with their previous platform. Qualitative data was collected using document analysis of key company documents and Focus Group Discussions (FGDs) with employees from ERPCo to explore and confirm different topics and themes. These methods were used to iteratively analyse the As-Is and To-Be situations at ERPCo and to develop and evaluate the proposed WLA Toolkit. The method used to analyse the collected data from the FGDs was the Qualitative Content Analysis (QCA) method. To develop the first component of the toolkit, the Organisation component, the organisational success factors that influence the success of WLA were identified using a Systematic Literature Review (SLR). These factors were discussed and validated in two exploratory FGDs held with employees from ERPCo, one with operational stakeholders and the other with strategic decision makers. The DeLone and McLean Information Systems (D&M IS) Success Model was used to undergird the research as a theory to guide the understanding of the factors influencing the success of WLA. Many of the factors identified in theory were found to be prevalent in the real-world-context, with some additional ones being identified in the FGDs. The most frequent challenges highlighted by participants were related to visibility; readily available high-quality data; flexibility of reporting; complexity of reporting; and effective decision making and insights obtained. Many of these related to the concept of usability issues for both the system and the information, which is specifically related to System Quality or Information Quality from the D&M IS Success Model. The second and third components of the toolkit are the Technology and Applications; and Information components respectively. Therefore, architecture and data management challenges and requirements for these components were analysed. An appropriate WLA architecture was selected and then further customised for use at ERPCo. A third FGD was conducted with employees who had more technical roles in ERPCo. The purpose of this FGD was to provide input on the architecture, technologies and data management challenges and requirements. In the Technology and Applications component of the WLA Toolkit, factors influencing WLA success related to applications and visualisations were considered. An instantiation of this component was demonstrated in the fourth FGD, where learning data from the LXP at ERPCo was collected and a dashboard incorporating recommended visualisation techniques was developed as a proof of concept. In this FGD participants gave feedback on both the dashboard and the toolkit. The artefact of this research is the WLA Toolkit that can be used by practitioners to guide the planning and implementation of WLA in large organisations that use LXP and WLA platforms. Researchers can use the WLA Toolkit to gain a deeper understanding of the required components and factors for successful WLA in software vendors. The research also contributes to the D&M IS Success Model theory in the information economy. In support of this PhD dissertation, the following paper has been published: Whale, A. & Scholtz, B. 2022. A Theoretical Classification of Organizational Success Factors for Workplace Learning Analytics. NEXTCOMP 2022. Mauritius. A draft manuscript for a journal paper was in progress at the time of submitting this thesis. , Thesis (PhD) -- Faculty of Science, School of Computer Science, Mathematics, Physics and Statistics , 2024
- Full Text:
- Date Issued: 2024-04
Enhanced visualisation techniques to support access to personal information across multiple devices
- Authors: Beets, Simone Yvonne
- Date: 2014
- Subjects: Information visualisation , Database management , Web services , Personal information management
- Language: English
- Type: Thesis , Doctoral , PhD
- Identifier: vital:10500 , http://hdl.handle.net/10948/d1021136
- Description: The increasing number of devices owned by a single user makes it increasingly difficult to access, organise and visualise personal information (PI), i.e. documents and media, across these devices. The primary method that is currently used to organise and visualise PI is the hierarchical folder structure, which is a familiar and widely used means to manage PI. However, this hierarchy does not effectively support personal information management (PIM) across multiple devices. Current solutions, such as the Personal Information Dashboard and Stuff I’ve Seen, do not support PIM across multiple devices. Alternative PIM tools, such as Dropbox and TeamViewer, attempt to provide a means of accessing PI across multiple devices, but these solutions also suffer from several limitations. The aim of this research was to investigate to what extent enhanced information visualisation (IV) techniques could be used to support accessing PI across multiple devices. An interview study was conducted to identify how PI is currently managed across multiple devices. This interview study further motivated the need for a tool to support visualising PI across multiple devices and identified requirements for such an IV tool. Several suitable IV techniques were selected and enhanced to support PIM across multiple devices. These techniques comprised an Overview using a nested circles layout, a Tag Cloud and a Partition Layout, which used a novel set-based technique. A prototype, called MyPSI, was designed and implemented incorporating these enhanced IV techniques. The requirements and design of the MyPSI prototype were validated using a conceptual walkthrough. The design of the MyPSI prototype was initially implemented for a desktop or laptop device with mouse-based interaction. A sample personal space of information (PSI) was used to evaluate the prototype in a controlled user study. The user study was used to identify any usability problems with the MyPSI prototype. The results were highly positive and the participants agreed that such a tool could be useful in future. No major problems were identified with the prototype. The MyPSI prototype was then implemented on a mobile device, specifically an Android tablet device, using a similar design, but supporting touch-based interaction. Users were allowed to upload their own PSI using Dropbox, which was visualised by the MyPSI prototype. A field study was conducted following the Multi-dimensional In-depth Long-term Case Studies approach specifically designed for IV evaluation. The field study was conducted over a two-week period, evaluating both the desktop and mobile versions of the MyPSI prototype. Both versions received positive results, but the desktop version was slightly preferred over the mobile version, mainly due to familiarity and problems experienced with the mobile implementation. Design recommendations were derived to inform future designs of IV tools to support accessing PI across multiple devices. This research has shown that IV techniques can be enhanced to effectively support accessing PI across multiple devices. Future work will involve customising the MyPSI prototype for mobile phones and supporting additional platforms.
- Full Text:
- Date Issued: 2014