# Welcome to Eduplex This is a web containing the documentation from Eduplex projects. You can choose an specific project on the navbar selector. {% content-ref url="/spaces/x3No2kOMD3vrsHDQpPoD" %} [NAPIL°AI-System](https://docs.eduplex.eu/adaptive-didaktische-systeme/) {% endcontent-ref %} NAPIL°AI enables modular, personalized learning solutions with real-time course creation based on individual needs. *** {% content-ref url="/spaces/gk6LhvMj50apP01XVAEk" %} [Adaptive Lernkonzepte](https://docs.eduplex.eu/adaptive-lernkonzepte/) {% endcontent-ref %} This section presents all conceptual developments related to adaptive learning approaches. *** {% content-ref url="/spaces/lqYnGAeG84PhHf0GNMZ3" %} [KI-gestütztes personalisiertes Lernen ("KI-Tutor")](https://docs.eduplex.eu/ki-gestutztes-personalisiertes-lernen-ki-tutor/) {% endcontent-ref %} One of the key developments in the project was AI-supported learning with an AI tutor. The results related to this can be found in this section. *** {% content-ref url="/spaces/ZYLS7UYNjVVnKXhY9htE" %} [Publikationen](https://docs.eduplex.eu/publikationen/) {% endcontent-ref %} An overview of all publications resulting from the project can be found here. *** {% content-ref url="/spaces/Xc6NwBLMjPv5DsJkfPPb" %} [Scorm to xAPI wrapper](https://docs.eduplex.eu/scorm-to-xapi-wrapper/) {% endcontent-ref %} Enabling seamless SCORM-to-xAPI conversion for LRS compatibility *** {% content-ref url="/spaces/rvgFTZLxwoOdoT8U9d3a" %} [User behavior tracker](https://docs.eduplex.eu/user-behavior-tracker/) {% endcontent-ref %} Captures user interactions and converts them into xAPI statements, enabling detailed, privacy-conscious analytics in LRS *** {% content-ref url="/spaces/w7BrZfWkVncWel7jBN9T" %} [Playlist builder UI](https://docs.eduplex.eu/nuxt-listing/) {% endcontent-ref %} The [Playlist Builder UI](https://docs.eduplex.eu/nuxt-listing/) provides components for creating, managing, and displaying customizable playlists of learning content. *** {% content-ref url="/spaces/KL3nFZnzySMpVOVZ9Bg1" %} [SSI Edu Wallets](https://docs.eduplex.eu/ssi-edu-wallets/) {% endcontent-ref %} SSI EduWallets enable secure, decentralized credential management, leveraging SSI standards for privacy, interoperability, and efficiency. *** {% content-ref url="/spaces/PnUjsAMcXqV3uh7MX1l3" %} [Recommendation, reporting & analytics](https://docs.eduplex.eu/text-analysis/) {% endcontent-ref %} The [Recommendation Engine](https://docs.eduplex.eu/text-analysis/) matches personalized learning opportunities with user profiles, using text analysis and ESCO taxonomy. The [Predictive Analytics](https://docs.eduplex.eu/text-analysis/reporting-and-predictive-analytics) project developed visualizations to interpret user metrics, helping optimize content delivery and engagement. The [LRS User Journey Visualizer](https://docs.eduplex.eu/text-analysis/lrs-user-journey-visualizer) project aimed to visualize user journeys facing challenges to provide effective visualization *** {% content-ref url="/spaces/CbjjMQ3k6WBKRmsyOs7t" %} [OpenSearch](https://docs.eduplex.eu/opensearch/) {% endcontent-ref %} Enable scalable vector search across indices for learning opportunities and skills *** {% content-ref url="/spaces/7EZ2jjZDgCMNNeg0ps99" %} [Skill Assessment Builder](https://docs.eduplex.eu/skill-assessment-tool/) {% endcontent-ref %} Comprehensive system for creating quizzes and assessing knowledge. *** {% content-ref url="/spaces/NEYP4EBHYjByLGQytZV2" %} [LTI Connector](https://docs.eduplex.eu/lti-connector/) {% endcontent-ref %} The LTI Connector enables seamless integration of LMS platforms with external tools using LTI standards. # How to export documentation ## Pandoc Using GitBook, all our documentation is in **markdown** format. When we want to export it in another formar we use [Pandoc](https://pandoc.org/), a command line tool which allows to convert from one format to another. On their website there is a [guide](https://pandoc.org/getting-started.html) on how to install it and how to use it. Pandoc allows to convert from markdown to many formats but currently we are using **markdown** to **docx**. ## Lua filters There is a problem with images because in markdown they can be in HTML format, GitBook also inserts images in HTML, but Pandoc is not able to parse HTML images inside markdown files. So we have created a [Lua filter](https://pandoc.org/lua-filters.html) to parse those HTML images into a valid format for Pandoc to be able to transform them into the desired format. ``` function convertHtmlImageToMarkdownImage (elem) if elem.tag == "RawInline" and elem.format == "html" and elem.text:match '% *** **KI Prompting Guide zur Lernmittelerstellung** Download als Lernmodul im xAPI-Format: {% file src="/files/CB0SykBNeENipkMiBpvR" %} Download als pdf: {% file src="/files/ZaX0FtxTex2V9Ty8NcDt" %} Die Inhalte sind zusätzlich auf der prototypischen Plattform bis mind. 31.12.2025 unter folgendem Link abrufbar: # 2.3 Kollaboratives Lernen Für kollaborative Lernszenarien wurde im Projekt ein innovatives und praxisnahes Lernformat in Form eines sogenannten ***Business Action Games*** entwickelt. Dieses stellt nicht klassisch das Spiel, sondern vielmehr die Simulation realer Arbeitssituationen ins Zentrum und fördert so gezielt Future Skills und überfachliche Kompetenzen. Die Teilnehmenden bearbeiten dabei über einen bestimmten Zeitraum hinweg täglich praxisorientierte Mikro-Challenges im Team, erhalten dazu passende Tools und Lerninhalte und reichen als Ergebnis kurze Video-Reflexionen ein, die Peer-Feedback auslösen. Der tägliche Aufwand für den einzelnen Lernenden beträgt dabei ca. 1 Stunde. Die kollektive Erfahrung, der tägliche Austausch sowie die Feedbackkultur stehen dabei im Vordergrund. Didaktisch wird das Konzept durch folgende technologischen Entwicklungen gestützt, welche im EduPLEx\_API-Prototypen implementiert wurden: · Standardisierung von Lerndaten · Dashboards für Learning Analytics · Skill-Profil-Erstellung über Feedback · Dynamische Peer-Zuordnung · Integration von Gamification-Elementen Ein zentrales Element ist die tägliche Feedback-Routine: Jeder Lernende bewertet zwei zufällig zugewiesene Peer-Beiträge, was nicht nur zur Entwicklung eigener Feedback-Kompetenz beiträgt, sondern zugleich wertvolle Daten für die kontinuierliche Weiterentwicklung der individuellen Future Skill Map liefert. Diese visualisiert die eigene Kompetenzentwicklung und erlaubt den Vergleich zwischen Selbst- und Fremdeinschätzung. Das Konzept wurde im Spiel „Team Trouble – das kollaborative Business Action Game“ prototypisch als 5-tägiges Teaser-Programm umgesetzt und erfolgreich in unsere LXP integriert. Ein vollständiger Spieldurchlauf wurde im Rahmen des **INVITE Tool-Check 2.0** (März 2024) durchgeführt und zeigte eine hohe Akzeptanz und positives Feedback. # Anhänge **Einführungsmodul für Business-Action Games** *** Download Lernmodul als xAPI-Format {% file src="/files/DhZfRN2K2qE5uHgWMFGn" %} Download Inhalte des Lernmoduls als pdf: {% file src="/files/q7yqkHaP3VC45Q7WUe43" %} Die Inhalte sind zusätzlich auf der prototypischen Plattform bis mind. 31.12.2025 unter folgendem Link abrufbar: # Definition Das Adaptive Didaktische System NAPIL°AI (Akronym für **N**euro**A**daptive **P**ersönlichkeits-**I**ntegrative **L**ernlösungen) ermöglicht eine modulare Konfiguration, Produktion und Begleitung von individuellen Lernlösungen. Damit ist neben Personalisierung/Individualisierung bei der Auswahl bereits vorhandener Kurse künftig eine Echtzeiterstellung von individuellen Lernlösungen möglich, die auf persönlichen Lernbedürfnissen basiert. # Hintergrund Da zu Projektstart das didaktische Handlungsfeld, Adaptives Lernen zu ermöglichen, noch nicht wirklich existent war und daher auch keine Systematik, Strukturen oder Standards vorlagen, lag die Entscheidung nahe, hier Grundlagenforschung und -entwicklung zu leisten. Daher ist neben der Entwicklung der 3 ursprünglich geplanten Lernszenarien mit dem NAPIL°AI System ein Adaptives Didaktisches System inklusive Lerntheorie und einer Lernmethodik entstanden, die fachliche Entwicklung und Persönlichkeitsentwicklung verbindet. Mit Blick auf technische Weiterentwicklungen fungiert das NAPIL°AI System als RAG-Quelle und didaktische Schaltstelle für das Auswerten von Lernerdaten, indem die Fachlogik didaktischen Handelns und dessen Rahmenbedingungen systematisch auf Termini, Konzepte und Zusammenhänge komprimiert wurde, die regelbasiert in eine Programmiersprache überführt werden können. # Forschungsansätze Das NAPIL°AI System ist ein Adaptives Didaktisches System, das eine Lerntheorie sowie Komponenten zur Konfiguration und Produktion von individuellen Lernlösungen umfasst und kombiniert erstmalig fachliche Entwicklung und Persönlichkeitsentwicklung. Damit lassen sich Lernlösungen im Baukastenprinzip gezielt nach individuellen Anforderungen entwickeln. Dies geschieht in der Startphase manuell. Künftig können Lernlösungen in Echtzeit durch Didaktische Intelligenz produziert werden, wenn die NAPIL°AI DIDAKTIK als didaktisches Regelwerk für Machine Learning z.B. via RAG Methode mit Sprachmodellen, Klassifizierungsmodellen, Vorhersagemodellen, Empfehlungsmodellen und weiteren benötigten emergenten KI-Modellen verknüpft wird. Dafür wurde die Fachlogik didaktischen Handelns und dessen Rahmenbedingungen systematisch auf Termini, Konzepte und Zusammenhänge komprimiert, die regelbasiert in eine Programmiersprache überführt werden können. Damit ist das NAPIL°AI System künftig zugleich Schaltstelle für das systematische didaktische Auswerten von Lernerdaten. Feintraining erfolgt fortlaufend u.a. durch Einspeisen der manuell konfigurierten Produkte und Abgleich von Mustern mit der Lerntheorie und daraus abgeleiteten Hypothesen. ![](https://3447609058-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fx3No2kOMD3vrsHDQpPoD%2Fuploads%2FAfuZl4iZVO3UHU7qMynm%2F0.png?alt=media) Abbildung 1: NAPIL°AI System – Überblick. Folgende Anforderungen waren leitend bei der Entwicklung der Lerntheorie: * Lerntheorie statt Lehrtheorie, wenn Lerner statt Provider im Zentrum des Lernprozessmanagements steht * Scope: Lernbedarfe (Fachentwicklung) & Lernbedürfnisse (Persönlichkeitsentwicklung) abdecken * Scope: Lernbedürfnisse im Rahmen von konstruktivistischen bis instruktionistischen Theorien abdecken * Didaktisches Handeln systemisch ermöglichen (zielgruppenübergreifend, rechtskreisübergreifend, plattformübergreifend, didaktisch ebenenübergreifend) * Adaptives System, das systematische Deduktion von Lernlösungen anterior aus gemeinsamer Basis ermöglicht, statt posterior singuläre Lösungen zusammenzuführen (Vermeidung von Lücken in didaktischer Logik & Folgefehlern in technischer Umsetzung) Die Konfiguration von individuellen Lernlösungen erfolgt durch Auswahl der entsprechenden Parameter aus dem Framework (basale, inhaltsbezogene, organisatorische, wertebezogene, methodische & sozialitätsbezogene Parameter; [Selbstbestimmtes Lernen - vgl. F. Peschel;](https://www.pedocs.de/volltexte/2019/17637/pdf/GSV_2002-Peschel-Qualitaetsmassstaebe_Hilfen_zur_Beurteilung_der_Offenheit_von_Unterricht.pdf)). Durch die methodische Modellierung, die Kompetenzorientierung ([vgl. 21st Century Skills/PISA](https://www.iqesonline.net/bildung-digital/digitale-schulentwicklung/modelle-zur-digitalisierung-von-schule-und-unterricht/das-4k-modell/)), Lernphasen ([vgl. L. Rosa](https://shiftingschool.wordpress.com/2014/09/23/lernen-zu-lehren-im-internetzeitalter/)) und Persönlichkeitsorientierung mit dem fachlichen Lerninhalt verknüpft, lassen sich individuelle Lernpräferenzen in die Produktion der Lernlösung einbinden. Die Modellierung basiert auf einem konzentrischen 3 Layer Prinzip kombiniert mit einer quaternären Segmentierung. Damit lässt sich ein Spektrum von 64 Kombinationen von Lernpräferenzen vordefinieren, die sowohl bei manueller als auch automatisierter Anwendung nachvollziehbar bleiben. Das Theorem der persönlichkeitsbezogenen Lernpräferenzen basiert auf dem interdisziplinären Ansatz, neurophysiologisch bedingte Reaktionsmuster und psychologische Grundbedürfnisse kausal zusammenzuführen ([vgl. J. LeDoux;](https://www.dasgehirn.info/denken/emotion/der-schaltkreis-der-angst) [G. Roth &](https://books.google.de/books?id=gUuOBAAAQBAJ\&printsec=frontcover\&hl=de\&source=gbs_ge_summary_r\&cad=0#v=onepage\&q\&f=false) [K. Grawe;](https://www.klaus-grawe-institut.ch/archiv/die-grundbeduerfnisse-nach-klaus-grawe-bei-kindern-und-jugendlichen-das-beduerfnis-nach-lust-und-unlust-1-4/)). Innerhalb einer neuentwickelten lerntheoretischen Taxonomie mit Kompetenzstufen und -leveln sind Lernzielverben dieser physiopsychologischen Strukturalität zugeordnet. Damit sind individuelle Lernpräferenzen bereits in der Konfigurationsphase systematisch implementierbar und mit einer Auswertbarkeit verknüpft. ![](https://3447609058-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fx3No2kOMD3vrsHDQpPoD%2Fuploads%2FIlHd5z1T7VTaBMB7Rss5%2F1.png?alt=media) Abbildung 2: NAPIL°AI System - Didaktisches Framework (Auszug). ![](https://3447609058-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fx3No2kOMD3vrsHDQpPoD%2Fuploads%2FtpoRMb29QCbkyLJltJiL%2F2.png?alt=media) Abbildung 3: NAPIL°AI System - Methodische Modellierung. ![](https://3447609058-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fx3No2kOMD3vrsHDQpPoD%2Fuploads%2FcRi4talPS0nOZRa19gsY%2F3.png?alt=media) Abbildung 4: NAPIL°AI System - Physiopsychologische Reaktionsmuster. ![](https://3447609058-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fx3No2kOMD3vrsHDQpPoD%2Fuploads%2FKVysHeq0ekVKKAq8Xi0Q%2F4.png?alt=media) Abbildung 5: NAPIL°AI System - Taxonomie der Lernzielverben (Auszug). Für das NAPIL°AI System wurden Parameter aller didaktischen Handlungsebenen (Meta-, Makro-, Meso-, Mikroebene) herausgearbeitet, mittels derer sich jede Lernlösung eindeutig innerhalb eines Didaktischen Frameworks identifizieren lässt. Dabei kann die Lernlösung wie bei klassischen Kursen bereits im Portfolio des Bildungsanbieters vorhanden sein oder für eine individuelle Lernlösung in Moment of Need konfiguriert werden. Die Parameter bilden basale, inhaltsbezogene, organisatorische, wertebezogene, methodische und sozialitätsbezogene Anforderungen an didaktische Handlungsfelder bei der Produktion individueller Lernlösungen ab. Jede Parameterkonstellation ist dadurch einzigartig. Damit ist das klassische Prinzip einer Personaerstellung künftig nicht mehr notwendig. Die Produktion erfolgt im Sinne der Zukunft des Lernens nicht wie bei klassischer Produktentwicklung vor der Buchung der Lernlösung, sondern danach. Um hier eine zeitnahe Produktion zu unterstützen, wurden didaktisierte Bauteile entwickelt, die vom Adaptieren von Learning Journeys über das Adaptivieren von Abläufen der Lernsessions bis hin zum adaptiven Generieren von Lern- und Lernkontrollaufgaben entsprechend der methodischen Modellierung in den Lernsettings automatisieren. Das System kann zudem bestehende Lernangebote über das Parameterprinzip klassifizieren und hinsichtlich der Neuroadaption und Persönlichkeitsentwicklung ergänzen und erleichtert so den Übergang bei der Transformation von gegenwärtigem zu künftigem Lernen. Fachliche Skills sollten aus Forschungssicht gemäß dem Gemeinsamen Europäischen Referenzrahmen für “Schlüsselqualifikationen Lebenslangen Lernens” des Europäischen Rates ([GER 2019)](https://op.europa.eu/en/publication-detail/-/publication/297a33c8-a1f3-11e9-9d01-01aa75ed71a1/language-en) grobgegliedert werden, um Lernern das Auffinden von Lerninhalten auf der Plattform zu erleichtern. Diese umfassen: * Lese- und Schreibkompetenz * Mehrsprachigkeit * Mathematische, wissenschaftliche und technische Fähigkeiten * Digitale und technologiebasierte Kompetenzen * Soziale Kompetenz und Fähigkeit, neue Kompetenzen zu erwerben * Aktive Bürgerschaft * Unternehmerische Kompetenz * Kulturbewusstsein und kulturelle Ausdrucksfähigkeit. # Ableitungen aus dem NAPIL°AI System hinsichtlich prototypischer Lernszenarien In der folgenden Tabelle wird anhand prägnanter Parameter die Klassifizierung bestehender Lernangebote (hier: prototypische Lernszenarien des Projekts aus AP 5) in Bezug auf das NAPIL°AI System nachvollziehbar gemacht. Der Klassifizierung sind zur Demonstration Empfehlungen zugefügt, wie bestehende Kurskonzepte ergänzt werden, um die Kombination von fachlicher und persönlicher Entwicklung umzusetzen. Da die Produktion der Lernlösungen mithilfe der KI-Modelle sowohl initial (nur künftige Lernlösungen) als auch fortlaufend (sowohl künftige als auch bestehende Lernangebote) möglich ist, können die Empfehlungen bei bestehenden Lernangeboten durch Überführen in Lernaufgaben mithilfe der NAPIL°AI Taxonomie und des Lernaufgabengenerators nachträglich umgesetzt werden. Dies ist ein Lösungsansatz, die WBS-interne Transformation der Lernformate zu unterstützen. Aktuell werden fixe Gruppen in einem 6:4-Lernformat unterrichtet. Der Mehraufwand, hier durch Trainer individuelle Lernaufgaben für 25 Teilnehmer zu erstellen, die den Lernpräferenzen der einzelnen Lerner entsprechen, ist manuell nicht umsetzbar. Die Automatisierung nach dem NAPIL°AI System bietet dafür essenzielle Unterstützung. | Didaktische Parameter | **Selbstgesteuertes Lernen** | **Trainergestütztes Lernen** | **Kollaboratives Lernen** | | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Organisatorische Parameter | Gruppenkonstellation > Flexible Gruppe, Gruppengröße > Einzelsetting (1), Lernort > Lernraum > Online > Asynchron > Interne Bibliothek (ECampus) / Serverort > Arbeitsplatz / Device > PC, Lernzeit > Zeittyp > Individuell | Gruppenkonstellation > Fixe Gruppe, Gruppengröße > Trainingssetting (bis 30), Lernort > Lernraum > Online > Synchron > Virtueller Klassenraum (LS3D) / Serverort > Homeoffice / Device > Laptop, Lernzeit > Zeittyp > Vollzeit | Gruppenkonstellation > Rollierende Gruppe, Gruppengröße > Kollaboratives Setting (bis 12), Lernort > Lernraum > Online > Teilsynchron > Realgame (Evulix) / Serverort > Mobil / Device > Smartphone, Lernzeit > Zeittyp > Individuell | | Methodische Parameter | Start bei Lernphase Verinnerlichen (Medieninput); Kompetenzstufen 3/6 & 4/6. Lernphasen Veräußerlichen, Kommunizieren und Kollaborieren finden nicht statt. Empfehlung: Veräußerlichen und Kommunizieren kann durch AI FELLOW > Tutor übernommen werden. Kollaborieren kann durch Peer > Workmate übernommen werden. Gewichtung der Kompetenzstufen wird durch Assessment ermittelt. | Start bei Lernphase Verinnerlichen (Trainerinput), Lernphasen Veräußerlichen, Kommunizieren und Kollaborieren finden nicht explizit statt. Empfehlung: Veräußerlichen kann durch AI FELLOW > Tutor; Kommunizieren durch AI FELLOW > Tutor und Peer > Gruppe und Peer > Partner; Kollaborieren durch Peer > Gruppe und Peer > Partner übernommen werden. Gewichtung der Lernphasen und Kompetenzstufen wird durch Assessment ermittelt. | Start bei Lernphase Verinnerlichen (Medieninput), danach Lernphase Kollaborieren, danach Lernphase Kommunizieren. Kompetenzstufen sind nicht transparent, Lernphase Veräußerlichen findet nicht statt. Empfehlung: Lernphase Veräußerlichen kann durch AI FELLOW > Tutor übernommen werden. Gewichtung der Kompetenzstufen wird durch Assessment ermittelt. | | Sozialitätsbezogene Parameter | Lernbegleitung > AI FELLOW > Tutor | Lernbegleitung > Expert > Trainer & Peer > Gruppe & AI FELLOW > Tutor | Lernbegleitung > Expert > Mentor & Peer > Partner & AI FELLOW > Mentor | # Ausblick Durch das Adaptive Didaktische System lassen sich künftig Lernlösungen zielgruppenunabhängig produzieren. Dies wird bei dem Partner WBS TRAINING AG ab 2025 verstärkt als Unternehmensziel verfolgt. Lerner, die mit individuelle Lernlösungen lernen, sind durch die Einbindung persönlicher Lernpräferenzen erfolgreicher im Erreichen ihrer Lernziele. Zudem haben sie auf dem Arbeitsmarkt höhere Chancen gegenüber Mitbewerbern, wenn sich auf Zertifikaten nicht nur fachliche Entwicklungen wiederfinden, sondern auch persönliche Entwicklungen. Im Rahmen des ESF-Projekts WASNUP werden 2 Lernangebote auf Basis des NAPIL°AI Systems entwickelt (Blended Learning für ausländische Fachkräfte im Pflegebereich zur Anerkennung von Abschlüssen sowie Online Learning für Stammpersonal im Pflegebereich zur Qualifikation zum Integrationscoach). Die Formate verlangen unterschiedliche Freiheitsgrade in der Konzeptionierung und sind daher gut geeignet, die Leistungsfähigkeit des NAPIL°AI Systems zu verproben. # Vision Autonome Plattformen können durch integrierte Didaktik nach dem NAPIL°AI System alle angeforderten Lernszenarien selbst generieren und bei Bedarf realer Personen diese zubuchen. # 1. Einleitung und Kontext #### 1. Einleitung und Kontext Im Rahmen des Forschungsprojekts **Eduplex\_API** wurde das übergeordnete Ziel verfolgt, **adaptives Lernen** durch verschiedene technologische Ansätze zu fördern. Insbesondere mit dem Aufkommen leistungsfähiger **Large Language Models (LLMs)** – wie etwa ChatGPT, GPT-3.5 oder GPT-4 von OpenAI – eröffneten sich neue Potenziale, Lernende nicht mehr nur regelbasiert oder über eigens trainierte Machine-Learning-Modelle zu begleiten, sondern **in Echtzeit und hochgradig personalisiert** zu unterstützen. Vor diesem Hintergrund entstand die Idee eines **KI-gestützten Tutors**, der in Form eines Chatbots lernzielorientierte und inhaltsbezogene Fragen beantworten und Lernende bei Bedarf individuell coachen kann – insbesondere im sogenannten *moment of need*. Da weder das Projektteam noch potenzielle Nutzer:innen oder Stakeholder zu Beginn eine konkrete Vorstellung der Umsetzung hatten, wurde ein **iterativer prototypischer Entwicklungsprozess** angestoßen, um Anforderungen, Nutzungskontexte und technische Machbarkeit systematisch zu erheben und zu evaluieren. Das Projekt gliederte sich in zwei aufeinanderfolgende Phasen: * **Phase 1 – Entwicklung eines Prototyps** * **Phase 2 – Entwicklung eines MVP (Minimum Viable Product)** inklusive Pilotierung mit Fachexpert:innen und Lernenden Dieser Bericht dokumentiert vorrangig **Phase 1 – die Prototypisierung** – sowie die daraus gewonnenen Erkenntnisse. Die nachfolgende **Phase 2**, in der das MVP entwickelt und in einem realen Anwendungskontext getestet wurde, ist im **„Eduplex – Abschlussbericht KI-Tutor“** ausführlich dargestellt (siehe Anhänge). # 2. Zielsetzung des Teilprojekts und Vorgehensweise **Zielsetzung** Das Teilprojekt verfolgte das Ziel, den Einsatz eines KI-basierten Tutors im Bildungskontext konzeptionell und prototypisch zu erproben. Dabei stand insbesondere die Frage im Mittelpunkt, inwieweit ein auf **Large Language Models** basierender Chatbot Lernende gezielt und wirksam unterstützen kann. Die Zielsetzungen im Einzelnen: * **Prototypische Konzeption eines KI-Tutors** in Form eines Chatbots, der Lernende auf unterschiedlichen Ebenen begleitet – fachlich (z. B. bei inhaltlichen Fragen), methodisch (z. B. bei der Auswahl geeigneter Lernstrategien) sowie organisatorisch (z. B. bei der Strukturierung des Lernprozesses). * **Iterative Entwicklung eines ersten Minimum Viable Product (MVP)**, um zentrale technische, didaktische und organisatorische Anforderungen in einem realitätsnahen Umfeld zu testen, zu bewerten und weiterzuentwickeln. * **Validierung des Ansatzes im kontinuierlichen Austausch mit relevanten Stakeholdern** sowie potenziellen Nutzergruppen, insbesondere durch qualitative Rückmeldungen und praxisorientierte Testszenarien. **Vorgehensweise und Methode** Die Umsetzung des Teilprojekts orientierte sich an agilen und nutzerzentrierten Entwicklungsprinzipien: * Es wurde ein **Rapid-Prototyping-Ansatz** verfolgt, bei dem frühzeitig einfache, aber funktionale Prototypen entwickelt wurden. Diese wurden in kurzen, iterativen Rückkopplungsschleifen gemeinsam mit Stakeholdern überprüft, bewertet und gezielt weiterentwickelt. * Ergänzend wurde ein **User-Centered- bzw. Stakeholder-Centered-Design** angewendet. Dabei wurden von Beginn an Nutzerbedarfe, Erfahrungen und Erwartungen systematisch erhoben und in die Gestaltung des KI-Tutors integriert. Dies diente dazu, die Lösung möglichst nah an realen Nutzungsszenarien und Bedarfen auszurichten. Diese methodische Kombination ermöglichte es, frühzeitig fundierte Rückmeldungen zu erhalten, die sowohl in die technische Umsetzung als auch in die didaktisch-methodische Ausgestaltung des MVP einflossen. # 3. Methodische Schritte im Überblick **Phase 1** des Projekts gliederte sich in mehrere methodische Schritte, die im Folgenden überblicksartig dargestellt werden: **1. Skizzierung von Wireframes und Informationsbedarfen** * Erste Entwürfe einer funktionalen Benutzeroberfläche wurden direkt in der No-Code-Entwicklungsumgebung *Bubble* umgesetzt. * **Learning:** Für vergleichbare Vorhaben empfiehlt sich in der frühen Konzeptionsphase der Einsatz spezialisierter UI/UX-Design-Tools (z. B. *Figma*), um erste Gestaltungsideen strukturierter und schneller zu visualisieren. *Bubble* hingegen erwies sich für die spätere Implementierung funktionaler Logiken als sehr leistungsfähig. **2. Aufbau der Datenbankstruktur** * Erstellung von Tabellen, Datenfeldern und Beziehungen zwischen zentralen Entitäten (z. B. Kurseinheiten, KI-Einstellungen, Nutzerprofile). * Die strukturierte Datenmodellierung diente als Grundlage für alle weiteren Entwicklungsschritte.
**3. Know-how-Aufbau und API-Verprobung mit Large Language Models** * Durchführung erster Funktionstests mit verschiedenen LLM-Endpunkten (u. a. GPT-3.5) zur Beantwortung von Fachfragen. * Ziel war das Verständnis für die Integration von Sprachmodellen und die Identifikation technischer Grenzen (z. B. Token-Limitierungen, Temperature-Einstellungen, System-Prompts). **4. Implementierung eines Retrieval-Augmented Generation (RAG) Systems** * Früh zeigte sich, dass reines Prompting nicht ausreicht, um spezifische Fachinhalte zuverlässig abzubilden. * Zur Lösung wurde ein RAG-System implementiert, das mit einer Vektordatenbank (z. B. *Pinecone*) und Ähnlichkeitssuche (Cosine Similarity) arbeitete. **5. Feedbackschleifen mit Stakeholdern und Lernbegleitungen** * Durchführung erster Usability- und Qualitätstests der KI-generierten Antworten. * Iteratives Vorgehen: Das Feedback floss direkt in die Optimierung technischer Parameter (Tokenanzahl, Temperature, Prompting-Strategie, Streaming-Optionen) und die Verfeinerung des RAG-Systems (u. a. durch besseres Chunking und Metadatenmanagement) ein. **6. Pilotierung im Fachbereich Finanzbuchhaltung (WBS TRAINING AG)** * Mangels externem KMU-Partner wurde der Prototyp im realen Weiterbildungskontext der WBS TRAINING AG getestet. * **Erkenntnisse:** Die Benutzerfreundlichkeit wurde als hoch eingeschätzt. Jedoch zeigten sich Schwächen in der fachlichen Genauigkeit – insbesondere bei mathematisch geprägten Inhalten. * **Herausforderung:** Balance zwischen eingebettetem Fachwissen (über RAG) und generativem Wissen des LLMs. Zudem war GPT-3.5 zum damaligen Zeitpunkt weniger leistungsfähig als spätere Modelle (z. B. GPT-4). **7. Auswertung und Differenzierung der Einsatzebenen** * Aus den Tests ließen sich drei Ebenen ableiten, auf denen ein KI-Tutor wirksam agieren kann: * **Makro-Ebene:** Auswahl und Empfehlung geeigneter Lerninhalte * **Meso-Ebene:** Fachliche Unterstützung bei konkreten Fragen * **Mikro-Ebene:** Methodisch-didaktische Begleitung, z. B. durch individuelles Feedback
**8. Überführung in ein MVP und Pilotierung** * Erkenntnisse aus Phase 1 wurden in eine konsolidierte Anforderungsliste überführt. * Auf dieser Basis wurde ein nutzbares MVP in *Bubble* realisiert, das in einer weiteren Testphase mit Teilnehmenden (TN) evaluiert wurde. * Die Ergebnisse dieser Pilotierung sind im separaten Bericht **„KI-Tutor\@WBS – Abschlussbericht“** dokumentiert. *** **Fazit zu Phase 1** * Die Phase des Rapid Prototypings erwies sich als entscheidend, um technische, didaktische und organisatorische Anforderungen frühzeitig zu identifizieren und gezielt weiterzuentwickeln. * Der kombinierte Ansatz aus schnellen Iterationen und enger Stakeholder-Einbindung schuf eine belastbare Grundlage für die zielgerichtete Entwicklung des MVP in Phase 2. # 4. System- und Konzeptübersicht Im Rahmen des Vorprojekts wurden zentrale Systemkomponenten und konzeptionelle Grundlagen geschaffen, die in der zweiten Projektphase weiterentwickelt und produktiv eingesetzt wurden. Die folgenden Entwicklungen bildeten das technologische und funktionale Fundament für die spätere MVP-Pilotierung: **1. Backend-Struktur** * **Content- und Kursmanagement:** Aufbau einer hierarchischen Kursstruktur mit übergeordneten Lernangeboten und untergeordneten Lerneinheiten. * **Integration KI-relevanter Steuerungsparameter:** Speicherung und Verwaltung von Metadaten zur Konfiguration von Systemprompts (Start-/Endprompts), Bewertungsskalen, Tokenlängen, Temperature-Werten, Text-to-Speech-Optionen u. v. m. * **Seiten-Typisierung:** Definition verschiedener Seitentypen für Lernformate mit und ohne KI-Unterstützung, um differenzierte Nutzererfahrungen zu ermöglichen.

Ansicht für Lernelement-Verwaltung

**2. Benutzerverwaltung** * **Nutzerprofil-Management:** Einrichtung und Konfiguration individueller Profile (z. B. Spracheinstellungen, gewünschter Schwierigkeitsgrad). * **Perspektivisch adaptiv ausbaufähig:** Schnittstellen zur Anbindung an adaptive Lernsysteme wurden konzipiert, um künftig Lernverläufe und persönliche Lernziele dynamisch berücksichtigen zu können. **3. Nutzer-Frontend und erste UX-Skizzierung** * **Navigationsmenü:** Ermöglicht die Auswahl unterschiedlicher Lernangebote und -einheiten. * **Hauptbereich:** Darstellung von Lerninhalten sowie interaktiven, KI-gestützten Formaten wie z. B. „Check-In“, „Lerntagebuch“ oder „Quiz“. * **KI-Chatbot (Pop-up-Funktion):** Permanente, aber deaktivierbare Begleitfunktion zur Beantwortung lernzielbezogener Fragen im jeweiligen inhaltlichen Kontext. * **Feedbacksystem:** Nutzer:innen können die Qualität der KI-Antworten sowie der bereitgestellten Lerninhalte bewerten – eine wichtige Grundlage für Optimierung und Qualitätssicherung.

User-Ansicht

**4. Geplante Funktionalitäten** * **Lernstandsbasierte Rückmeldung:** Integration von Fortschrittsdaten, um Antworten der KI dynamisch an individuelle Lernstände anzupassen. * **Automatisierte RAG-Anbindung:** Zielgerichtete Verknüpfung mit Fachinhalten über ein Retrieval-Augmented Generation-System, um kontextspezifische und valide Informationen in Echtzeit bereitstellen zu können. > # 5. Herausforderungen und Lösungsansätze Trotz durchweg positiver Resonanz seitens der Stakeholder zeigte sich im Projektverlauf eine Reihe technischer, organisatorischer und didaktischer Herausforderungen. Im Folgenden werden zentrale Problemstellungen sowie erste Lösungsansätze dargestellt: **1. Unstrukturierte Datenlage** * **Problem:** Viele vorhandene Lerninhalte liegen in proprietären Formaten vor (z. B. PDF-Dateien, WBTs oder Articulate-Module), was die Extraktion und das Chunking der Inhalte für RAG-Systeme erheblich erschwert. * **Lösungsansatz:** Für den gezielten Zugriff auf Fachinhalte sind strukturierte Datenquellen (z. B. Glossare, CSV-Dateien oder reine Textformate) deutlich besser geeignet. Eine gezielte Aufbereitung vorhandener Materialien oder die Entwicklung KI-freundlicher Content-Formate kann die Grundlage für eine leistungsfähige RAG-Anbindung schaffen. **2. Dynamische Einbettung von Prompting-Daten** * **Problem:** Um personalisierte Lernerlebnisse zu ermöglichen, müssen KI-Prompts flexibel auf Nutzereigenschaften wie Sprache, Lernstand oder Präferenzen reagieren können. Dies erfordert die Verarbeitung sowohl statischer als auch dynamischer Daten. * **Lösungsansatz:** Die Entwicklung adaptiver Prompt-Strategien und der Aufbau technischer Schnittstellen zur Nutzerprofilintegration (z. B. über APIs) sind essenziell. Die dynamische Steuerung von Systemprompts kann so individualisierte Lerndialoge ermöglichen. **3. Begrenzte Verlässlichkeit aktueller LLMs** * **Problem:** Bei komplexen oder fachlich spezialisierten Themenfeldern (z. B. Finanzbuchhaltung) lieferten Sprachmodelle vereinzelt fehlerhafte oder unpräzise Informationen. * **Lösungsansatz:** Kurzfristig hilft eine gezielte RAG-Anbindung mit geprüften Inhalten, um die Zuverlässigkeit zu erhöhen. Langfristig wird die Qualität maßgeblich durch die Weiterentwicklung leistungsfähiger Modelle wie GPT-4 und folgende Generationen bestimmt bleiben. Dennoch bleibt die Prüfung von KI-generierten Inhalten durch Fachexpert:innen ein kritischer Faktor. **4. Change-Prozess und Integration der Stakeholder** * **Problem:** Die Einführung KI-gestützter Lernsysteme stellt eine tiefgreifende Veränderung dar, die ohne begleitende Maßnahmen auf Widerstand bei Lehrenden, Fachexpert:innen und Lernenden stoßen kann. * **Lösungsansatz:** Der Change-Prozess sollte frühzeitig durch partizipative Formate begleitet werden. Die gemeinsame Entwicklung konkreter Use Cases und eine transparente Kommunikation der Potenziale fördern Akzeptanz und Nutzungsbereitschaft. **5. Unklare Wirkung auf den Lernerfolg** * **Problem:** Der tatsächliche Einfluss eines KI-Tutors auf den individuellen Lernerfolg konnte im Projektverlauf noch nicht umfassend empirisch belegt werden. * **Lösungsansatz:** Für eine belastbare Evaluation sind klar definierte Lernziele, geeignete Messinstrumente und ein systematisches Monitoring erforderlich. Die Nutzung von Analyse-Standards wie xAPI kann helfen, differenzierte Erkenntnisse über Lernverläufe, Interaktionen und die Wirksamkeit der KI-Unterstützung zu gewinnen. # 6. Erkenntnisse und mögliche Weiterentwicklung Das Vorprojekt legte zentrale technische und didaktisch-organisatorische Grundlagen für den weiteren Ausbau eines KI-Tutors. Im Rahmen der Umsetzung konnten wertvolle Erkenntnisse gewonnen werden, die nachfolgend systematisiert dargestellt sind: **1. Vielfältige Einsatzmöglichkeiten des KI-Chatbots** * Der KI-Tutor agiert nicht nur als klassischer *Frage-Antwort-Assistent*, sondern kann auch bei **methodischer Unterstützung**, **Motivationsförderung** und **Selbstreflexion** hilfreich sein – z. B. durch Lerntagebücher, Quiz-Formate oder Monitoring des Lernfortschritts. * Die Einteilung in **Makro-, Meso- und Mikro-Ebene** hat sich als hilfreiches Strukturierungsinstrument für Use Cases erwiesen. **2. Abwägung zwischen automatisierter Fachexpertise und RAG-Einbindung** * Der Einsatz eines **RAG-Systems** (Retrieval-Augmented Generation) oder vergleichbarer Mechanismen ist essenziell, um verlässliche Fachinformationen bereitzustellen und typische **„Halluzinationen“ generativer Modelle** zu vermeiden. * Je nach Fachgebiet ist eine **sorgfältige inhaltliche Aufbereitung** (Chunking, semantische Strukturierung, Metadatenmanagement) unerlässlich. **3. Iteratives Stakeholder-Feedback als Erfolgsfaktor** * Kurze Entwicklungszyklen und regelmäßige Prototyp-Tests mit Stakeholdern haben sich als wirkungsvoll erwiesen, um Akzeptanz zu fördern und Bedarfe realistisch abzubilden. * Insbesondere in Bildungskontexten ist es entscheidend, **Lehrende und Lernende aktiv in die Gestaltung und Erprobung** einzubinden. **4. Perspektiven der Integration in bestehende Lernsysteme** * Die langfristige **Anbindung an Learning-Management-Systeme (LMS)** und **Learning-Record-Stores** (z. B. über xAPI) stellt einen bedeutenden Mehrwert dar. * Dadurch können **individualisierte Lernpfade** und ein **adaptives Feedbacksystem** realisiert werden, das den Lernstand gezielt berücksichtigt. *** #### ❓ Offene Fragestellungen aus der prototypischen Entwicklung * **Wie kann der Lerneffekt eines KI-Tutors valide und differenziert evaluiert werden?**\ (z. B. Definition geeigneter Metriken, Kombination qualitativer und quantitativer Verfahren) * **Wie lassen sich Datenschutz und Datensicherheit gewährleisten,**\ insbesondere bei der Nutzung externer APIs wie z. B. OpenAI? * **Wie kann ein KI-Tutor langfristig skaliert werden?**\ (z. B. Übertragbarkeit auf größere Lernsettings, weitere Fachgebiete oder Zielgruppen) *** Diese offenen Punkte bilden zentrale Entwicklungsfelder für **Phase 2** und darüber hinausgehende Projektabschnitte. Eine systematische Bearbeitung ist erforderlich, um das Gesamtkonzept zu validieren, wissenschaftlich zu fundieren und für den breiten Einsatz weiterzuentwickeln. # 7. Ausblick und Empfehlungen Das Vorprojekt hat wesentliche Grundlagen für die Entwicklung eines robusten, adaptiven und didaktisch fundierten KI-Tutoring-Systems gelegt. Die Ergebnisse bestätigen die Relevanz und Machbarkeit eines solchen Ansatzes im Bildungskontext. Um das Potenzial weiter auszuschöpfen, empfehlen sich für die nächsten Schritte folgende Entwicklungsrichtungen: **1. Weiterentwicklung des MVP** * **Ausbau des RAG-Systems** durch den Einsatz fortschrittlicherer Vektordatenbanken und die gezielte Anreicherung mit feingranulierten Metadaten. * **Optimierung der Facheinbindung**, insbesondere bei komplexen oder stark formalisieren Themenfeldern wie Finanzbuchhaltung, Recht oder MINT-Fächern. **2. Didaktische Konzepte** * **Entwicklung strukturierter KI-Lernszenarien**, die die Stärken eines Chatbots – wie On-Demand-Erklärungen, personalisierte Lernbegleitung oder niederschwelligen Zugang – systematisch mit Lernzielen und didaktischen Modellen verknüpfen. * **Erprobung verschiedener Interaktionsformen** wie textbasierte Dialoge, gesprochene Kommunikation oder Gamification-Ansätze zur Steigerung der Motivation und Nutzungsakzeptanz. **3. Qualitätssicherung** * **Kontinuierliches Monitoring der KI-Antworten**, kombiniert aus automatisierten Mechanismen (z. B. Confidence-Scoring, Prüfalgorithmen) und manuellen Prüfverfahren (z. B. Freigabelogiken). * **Einbindung von Selbstreflexionsmechanismen innerhalb der KI**, z. B. durch gezielte Prompting-Strategien wie *Chain-of-Thought* oder *Re-Checking Queries*, um die Qualität der Antworten proaktiv zu erhöhen. **4. Langzeitstudien zum Lernerfolg** * **Durchführung empirisch fundierter Langzeitstudien** mit definierten Pilotgruppen, um den tatsächlichen Lerneffekt eines KI-Tutors sowohl quantitativ als auch qualitativ nachweisen zu können. * **Einbeziehung heterogener Zielgruppen** (z. B. Jugendliche, Berufstätige, ältere Lernende), um die Anwendbarkeit und Skalierbarkeit des Ansatzes für unterschiedliche Bildungskontexte zu evaluieren. **5. Schaffung strukturierter Lerninhalte** * **Förderung von Standards in der Content-Erstellung**, insbesondere durch den gezielten Einsatz geeigneter E-Learning-Authoring-Tools, um spätere technische Einbettung zu erleichtern. * **Entwicklung von Best-Practice-Beispielen** für Content-Strukturierung, Chunking-Strategien und Metadatenverwaltung zur Unterstützung der RAG-Integration. *** Mit diesen Weiterentwicklungen kann das Potenzial eines KI-basierten Tutors nicht nur ausgeschöpft, sondern auch systematisch in bestehende Bildungslandschaften integriert werden – als wirksames Instrument für personalisiertes, selbstgesteuertes und adaptives Lernen. # 8. Zusammenfassung Das Teilprojekt **„Prototypische Skizzierung eines KI-Tutors“** hat gezeigt, wie ein KI-gestütztes Lernsystem konzipiert und in kurzer Zeit prototypisch umgesetzt werden kann. Dabei wurden sowohl technologische als auch didaktisch-methodische Erkenntnisse gewonnen, die eine solide Grundlage für die Weiterentwicklung des Gesamtprojekts bilden. **Zentrale Erfolgsfaktoren waren:** * **Iteratives Vorgehen mit kurzen Entwicklungszyklen**: Die agile Arbeitsweise mit kontinuierlichem Stakeholder-Feedback ermöglichte schnelle Anpassungen und eine nutzernahe Entwicklung. * **Technologisches Fundament**: Der Aufbau strukturierter Datenquellen, einer tragfähigen Backend-Architektur sowie die gezielte Einbindung von KI-APIs (z. B. OpenAI, RAG-Systeme) stellten die technischen Kernkomponenten dar. * **Frühe Realerprobung im Bildungskontext**: Die Integration praktischer Testphasen – insbesondere im Kontext der WBS TRAINING AG – sowie die enge Zusammenarbeit mit Lehrenden und Lernenden führten zu realitätsnahen Use Cases und belastbaren Anforderungen. **Herausforderungen und offene Fragen:** Trotz der positiven Entwicklung wurden auch zentrale Herausforderungen identifiziert, etwa: * **Datenqualität und Content-Strukturierung** als Voraussetzung für verlässliche RAG-Anbindungen. * **Fachgenauigkeit bei komplexen Themenbereichen** (z. B. Finanzbuchhaltung). * **Evaluation des tatsächlichen Lernerfolgs** sowie die langfristige **Akzeptanz** von KI-basierten Lernsystemen im pädagogischen Alltag. Diese Themen sind Gegenstand der weiteren Entwicklung in Phase 2 und darüber hinaus. **Fazit und Ausblick:** Die Erkenntnisse aus Phase 1 schaffen die konzeptionelle und technologische Basis, um ein **vollwertiges Minimum Viable Product (MVP)** zu realisieren und in konkreten Weiterbildungsszenarien zu erproben. Die in Phase 2 durchgeführte **Pilotierung und Evaluation** wird es ermöglichen, gezielte Optimierungen vorzunehmen – mit dem Ziel, einen **nachhaltigen und wirksamen KI-Tutor** für den Einsatz in der beruflichen Bildung zu etablieren. # 9. Anhänge **Abschlussbericht für MVP KI-Tutor (KI-gestützte Lernbegleitung):** {% file src="/files/zh3Y5YHcV5kFSXkJLpPy" %} **Handlungsempfehlungen & Best Practices für RAG-Systeme:** {% file src="/files/Ws2uRESH3TaSsTE45VUJ" %} {% file src="/files/JWIWaR79eLLhNAca429a" %} *** # Publikationen Englmeier, K. (2023) The Increasing Importance of Natural Language Interaction in the Design of Intelligent Tutoring Systems., In: Miguel Baptista Nunes, Pedro Isaías, Philip Powell (eds), Proceedings of the IADIS International Conference Information Systems 2023, Lisbon (2023) S. 13-20. {% file src="/files/xnSsXOnRdyFBb8xLaDjW" %} *** Englmeier, K., Contreras, P. (2023). Operationalising ontologies for competence management in the industry. In: Tareq Ahram and Redha Taiar (eds) Human Interaction and Emerging Technologies (IHIET-AI 2023): Artificial Intelligence and Future Applications. AHFE (2023) International Conference. AHFE Open Access, vol 70. AHFE International, USA. {% file src="/files/FlAfgMy9pHlXOYetAbd3" %} *** Englmeier, K. (2024). User-Centered Models for Adaptive Learner Journeys in Self-Paced Learning. Proceedings of the International Conference on Intelligent Human Systems Integration: Integrating People and Intelligent Systems (IHSI 2024), Università degli Studi di Palermo, February 22-24, 2024, Palermo, Italy. {% file src="/files/10ocoY8cAxvQZcBpjf7a" %} *** Englmeier, K. (2024). Implementation of Instructional Design for Self-Paced Learning, Proceedings of the International Multi Conference on Computer Science and Information Systems (MCCSIS) ’24, July 12-15, 2024 Budapest, Hungary. {% file src="/files/RkNiop80UdCQzF3g71gp" %} *** Englmeier, K. (2024). Insights Gained from Integrating Self-Paced Learning into Digital Learning Environments. In: Tareq Ahram, Waldemar Karwowski, Darko Etinger and Tea Mijač (eds) Human Systems Engineering and Design (IHSED2024): Future Trends and Applications. AHFE (2024) International Conference. AHFE Open Access, vol -2. AHFE International, USA. {% file src="/files/NmUlsTlXH9UanHFoGwKz" %} *** Englmeier, K., Contreras, P. (2024). How AI Can Help Learners to Develop Conceptual Knowledge in Digital Learning Environments. Proceedings of the International Conference IEEE Intelligent Systems IS’24, August 29-31, 2024, Varna, Bulgaria. Best Paper. {% file src="/files/dmTWjNL94gHtU8OsSZ43" %} *** Englmeier, K. (2024). The Design of Self-Paced Learning for Structured Learning Environments. Proceedings of the CENTERIS - International Conference on ENTERprise Information Systems, November 13-15, 2024, Funchal, Portugal. {% file src="/files/laFBGWmTmKQwZGIDi1Kn" %} # Scorm to xAPI wrapper ## Required files Main git repository with source code and working example: All required files are located in the src directory. * xapiwrapper.min.js - Manages complexities of the xAPI and includes the ADL core verbs (**This file must be loaded as a script in the page**). The following files should be imported to use their functions: * APIWrapper.util.js - This is the generic wrapper that calls functions from both scormApi12 and scormApi2004 wrapper objects. * baseAPI.util.js - This file contains code that is reused by scormApi12 and scormApi2004, and some constants used in APIWrapper. A manual import it is not needed, files which use it will automatically import it. * scormApi12.util.js - The SCORM 1.2 wrapper object. * scormApi2004.util.js - The SCORM 2004 wrapper object. * SCORMToXAPIFunctions.js - Contains all the code required to map SCORM data model elements to xAPI statements. ## Configuration You will need to set several configuration values in order for the wrappers to communicate with the LRS: ``` { "lrs":{ "endpoint":"https://lrs-eu-central-1.eduplex.eu/data/xAPI/", "user":"64b9634d139ecf5d77ec8bf335c582c8ee374f5", "password":"password1234" }, "courseId":"https://scorm.com/golf-example", "lmsHomePage":"http://lms.adlnet.gov", "isScorm2004":false, "activityId":"https://scorm.com/golf-example/activity/test", "groupingContextActivity":{ "definition": { "name": { "en-US": "My Golf Example" }, "description": { "en-US": "My Golf Example happening from 2022" } }, "id": "https://scorm.com/golf-example/demo/test", "objectType": "Activity" } } ``` The most important properties are: * `lrs.endpoint` Should define the URL where the LRS is located (it could be just the domain or it could include also a path prefix ending with `/`). * `lrs.user` *Username* or *access key* from the LRS client. * `lrs.password` *Password* or *secret* from the LRS client. * `isScorm2004` Should define if the SCORM 2004 or SCORM 1.2. * `activityId` Unique identifier of this SCORM package in the LRS. * `courseId` The [IRI](https://www.ietf.org/rfc/rfc3987.txt) of the entire course activity. This will be used as context to group statements. * `groupingContextActivity.id` It can be the same as `activityId`. ## Try it yourself You can clone the repository to test yourself the functionality. Keep in mind you will need a web server serving all files, otherwise the browser will block requests due to CORS. You need the whole repository as it is in order to make the example files work correctly. You would need to add your LRS configuration and credentials in the function called **getXapiConfig** from the file `demo/index.js`. After that you are ready to try it. Access **index.html** which will load a simple SCORM package (already part of this project) where you can do some clicks and answer a test. With each interaction you will be sending data to the LRS as xAPI *statements* that you can check in your LRS dashboard. The *state* of the progress will also be stored in the LRS. You can replace the *ExampleScorm* package with any other package of your choice, just remember to unzip the original package and use the same directory name. ## Other considerations * It is important the iframe with the SCORM is loaded after all the javascript code is executed, all window objects are created and event listeners are set. * This project implementation results in a lot of HTTP requests each time there is an interaction from the user. To reduce this amount we added a delay (implemented with `setTimeout` in `demo/index.js`) in the event that triggers the request to update the attempt state in the LRS. But this is not strictly necessary in order to work. * The versions of SCORM supported are 1.2 and 2004. Make sure to provide the right version through **isScorm2004** field in the xAPI configuration object and also in the **init** method. * This test repository was tested with [Learning Locker](https://github.com/LearningLocker/learninglocker) ## How does it work In order to start tracking SCORM courses with xAPI statements first we need to store the LRS configuration, create some window objects which are required in the wrapper files and set some event listenners to convert SCORM actions to xAPI. Most of this objects and functions are provided by the files mentioned in the first section of this project. You will only need to provide some small configuration or make changes in the behaviour of the events if you want to add extra functionality. You will just need to call this init method **before loading your SCORM package** on an iframe. You can check an example of the implementation on the file `demo/index.js`. In this file would be important to change the following variables: * `isScorm2004` Depending on the version used: SCORM 2004 or SCORM 1.2 * `actorId` Unique identifier for the learner/student The idea of this file is to overwrite some needed values and implement some event handlers. This is just a proof of concept, but adapting this code into any platform or application should be easy. It has been proven to work on an SPA (single page application) ## License The source code for the site is licensed under the [**MIT license**](https://gitlab.com/eduplex-api), which you can find in the [LICENSE](https://gitlab.com/eduplex-api/scorm-to-xapi-wrapper/-/blob/main/LICENSE/README.md) file. The ExampleScorm files are downloaded from the [SCORM - Golf Examples site](https://scorm.com/scorm-explained/technical-scorm/golf-examples/#run-time-examples) (SCORM 1.2 complete example from "Basic Run-Time Calls" section) by [Rustici Software LLC](http://www.scorm.com/). It is licensed under [Creative Commons Attribution 3.0 United States License](http://creativecommons.org/licenses/by/3.0/us/). All credits to the creator of the original demo content, no changes were made on this content. Files are based on another open source repository () which has both Apache License and Creative Commons. You can find more information about licenses inside each file. # Example score xAPI statements ## Answer question
Click to see full statement with answer and/or score ``` { "stored": "2023-07-12T15:30:24.576Z", "active": true, "completedForwardingQueue": [], "failedForwardingLog": [], "client": "62ed2d33caa1070008fbe17d", "lrs_id": "62ed2d33caa1070008fbe17c", "completedQueues": [], "activities": [ "http://1U1_qbiXXhuOAioSMG6xipmPd4-mFSfX_rise_v5/ksILhw_SMiaXQ9jUayJ0yH9mTd0ODnf2/cjyovbks400043b5yxqy0i9s9_1689175824374" ], "hash": "92ef650d473cc6d10d1c3cf57c5d075c1f2866c1", "agents": [ "https://proto.eduplex.eu|3" ], "statement": { "authority": { "objectType": "Agent", "name": "EduPLEx Client", "mbox": "mailto:email@example.com" }, "stored": "2023-07-12T15:30:24.576Z", "actor": { "objectType": "Agent", "account": { "name": "3", "homePage": "https://proto.eduplex.eu" } }, "timestamp": "2023-07-12T15:30:24.374Z", "version": "1.0.0", "id": "ab3556af-b987-40cc-ae9f-34bfc409f7aa", "result": { "success": false, "response": "cjyovbks400063b5ynrepy1rp" }, "verb": { "id": "http://adlnet.gov/expapi/verbs/answered", "display": { "en-US": "answered" } }, "object": { "id": "http://1U1_qbiXXhuOAioSMG6xipmPd4-mFSfX_rise_v5/ksILhw_SMiaXQ9jUayJ0yH9mTd0ODnf2/cjyovbks400043b5yxqy0i9s9_1689175824374", "objectType": "Activity", "definition": { "type": "http://adlnet.gov/expapi/activities/cmi.interaction", "name": { "und": "Take care when writing questions so that learners can interpret them correctly. Use simple, straightforward language." }, "description": { "und": "Take care when writing questions so that learners can interpret them correctly. Use simple, straightforward language." }, "interactionType": "choice", "correctResponsesPattern": [ "cjyovbks400053b5yi8r78s8l" ], "choices": [ { "id": "cjyovbks400053b5yi8r78s8l", "description": { "und": "Choice 1" } }, { "id": "cjyovbks400063b5ynrepy1rp", "description": { "und": "Choice 2" } }, { "id": "cjyovtgis00353b5yuf6kcai5", "description": { "und": "Choice 3" } }, { "id": "cjyovtl0900363b5yu7y5bojb", "description": { "und": "Choice 4" } }, { "id": "ckkz884nw002v3b6irh6j377p", "description": { "und": "sfdsdfdsfs" } } ] } } }, "hasGeneratedId": false, "deadForwardingQueue": [], "voided": false, "verbs": [ "http://adlnet.gov/expapi/verbs/answered" ], "processingQueues": [], "person": null, "timestamp": "2023-07-12T15:30:24.374Z", "relatedActivities": [ "http://1U1_qbiXXhuOAioSMG6xipmPd4-mFSfX_rise_v5/ksILhw_SMiaXQ9jUayJ0yH9mTd0ODnf2/cjyovbks400043b5yxqy0i9s9_1689175824374" ], "relatedAgents": [ "https://proto.eduplex.eu|3", "mailto:email@example.com" ], "organisation": "62ed1d89858c0c0008338e5c", "_id": "64aec7106d0a00001bf75e52", "registrations": [], "pendingForwardingQueue": [] } ```
## Finish questionnaire with score ### xAPI **passed** verb
Click to see xAPI passed verb ``` { "stored": "2023-07-12T15:34:20.924Z", "active": true, "completedForwardingQueue": [], "failedForwardingLog": [], "client": "62ed2d33caa1070008fbe17d", "lrs_id": "62ed2d33caa1070008fbe17c", "completedQueues": [], "activities": [ "http://1U1_qbiXXhuOAioSMG6xipmPd4-mFSfX_rise_v5" ], "hash": "47a58cd03db309912eb3b145fd28bdf789145d12", "agents": [ "https://proto.eduplex.eu|3" ], "statement": { "authority": { "objectType": "Agent", "name": "EduPLEx Client", "mbox": "mailto:email@example.com" }, "stored": "2023-07-12T15:34:20.924Z", "actor": { "objectType": "Agent", "account": { "name": "3", "homePage": "https://proto.eduplex.eu" } }, "timestamp": "2023-07-12T15:34:20.810Z", "version": "1.0.0", "id": "92319379-217d-4812-880f-9490fbbd34b3", "result": { "success": true, "duration": "PT5M58.724S", "completion": true }, "verb": { "id": "http://adlnet.gov/expapi/verbs/passed", "display": { "en-US": "passed" } }, "object": { "id": "http://1U1_qbiXXhuOAioSMG6xipmPd4-mFSfX_rise_v5", "objectType": "Activity", "definition": { "type": "http://adlnet.gov/expapi/activities/course" } } }, "hasGeneratedId": false, "deadForwardingQueue": [], "voided": false, "verbs": [ "http://adlnet.gov/expapi/verbs/passed" ], "processingQueues": [], "person": null, "timestamp": "2023-07-12T15:34:20.810Z", "relatedActivities": [ "http://1U1_qbiXXhuOAioSMG6xipmPd4-mFSfX_rise_v5" ], "relatedAgents": [ "https://proto.eduplex.eu|3", "mailto:email@example.com" ], "organisation": "62ed1d89858c0c0008338e5c", "_id": "64aec7fc6d0a00001bf75e5d", "registrations": [], "pendingForwardingQueue": [] } ```
### xAPI **progressed** verb
Click to see xAPI progressed verb ``` { "stored": "2023-07-12T15:38:55.728Z", "active": true, "completedForwardingQueue": [], "failedForwardingLog": [], "client": "62ed2d33caa1070008fbe17d", "lrs_id": "62ed2d33caa1070008fbe17c", "completedQueues": [], "activities": [ "http://1U1_qbiXXhuOAioSMG6xipmPd4-mFSfX_rise_v5" ], "hash": "725aa39b12ef25efb420f005991d8f8c2f3955bd", "agents": [ "https://proto.eduplex.eu|3" ], "statement": { "authority": { "objectType": "Agent", "name": "EduPLEx Client", "mbox": "mailto:email@example.com" }, "stored": "2023-07-12T15:38:55.728Z", "actor": { "objectType": "Agent", "account": { "name": "3", "homePage": "https://proto.eduplex.eu" } }, "timestamp": "2023-07-12T15:38:55.614Z", "version": "1.0.0", "id": "99af6f89-c753-46dc-9dd8-d0c5ef315896", "result": { "duration": "PT10M25.406S", "extensions": { "http://w3id.org/xapi/cmi5/result/extensions/progress": 100 }, "completion": true }, "verb": { "id": "http://adlnet.gov/expapi/verbs/progressed", "display": { "en-US": "progressed" } }, "object": { "id": "http://1U1_qbiXXhuOAioSMG6xipmPd4-mFSfX_rise_v5", "objectType": "Activity", "definition": { "type": "http://adlnet.gov/expapi/activities/course" } } }, "hasGeneratedId": false, "deadForwardingQueue": [], "voided": false, "verbs": [ "http://adlnet.gov/expapi/verbs/progressed" ], "processingQueues": [], "person": null, "timestamp": "2023-07-12T15:38:55.614Z", "relatedActivities": [ "http://1U1_qbiXXhuOAioSMG6xipmPd4-mFSfX_rise_v5" ], "relatedAgents": [ "https://proto.eduplex.eu|3", "mailto:email@example.com" ], "organisation": "62ed1d89858c0c0008338e5c", "_id": "64aec90f6d0a00001bf75e6e", "registrations": [], "pendingForwardingQueue": [] } ```
### H5P **scored** verb
Click to see H5P scored verb ``` { "stored": "2023-04-12T12:30:48.988Z", "active": true, "completedForwardingQueue": [], "failedForwardingLog": [], "client": "6079932d462e7205c3a5de21", "lrs_id": "6079932d462e7205c3a5de20", "completedQueues": [ "STATEMENT_PERSON_QUEUE", "STATEMENT_QUERYBUILDERCACHE_QUEUE", "STATEMENT_FORWARDING_QUEUE" ], "activities": [ "http://4198-46917_SCORM_2004 4th Edition" ], "hash": "143b4cc4ee072ff1630443dc4f8551bf791c7dfc", "agents": [ "https://www.courseticket.com|97774" ], "statement": { "authority": { "objectType": "Agent", "name": "leraninglocker-ct", "mbox": "mailto:email@example.com" }, "stored": "2023-04-12T12:30:48.988Z", "context": { "contextActivities": { "grouping": [ { "id": "http://4198-46917_SCORM_2004 4th Edition?attemptId=97774_a1", "objectType": "Activity", "definition": { "type": "http://adlnet.gov/expapi/activities/attempt" } }, { "id": "https://www.courseticket.com/de/user-service-detail/159724/e-learning/4198", "objectType": "Activity", "definition": { "type": "http://adlnet.gov/expapi/activities/course" } }, { "definition": { "name": { "en-US": "4198-46917_SCORM_2004 4th Edition" }, "description": { "en-US": "4198-46917_SCORM_2004 4th Edition" } }, "id": "http://4198-46917_SCORM_2004 4th Edition", "objectType": "Activity" } ], "category": [ { "id": "https://w3id.org/xapi/scorm", "objectType": "Activity" } ] } }, "actor": { "objectType": "Agent", "account": { "homePage": "https://www.courseticket.com", "name": "97774" } }, "timestamp": "2023-04-12T12:30:48.988Z", "version": "1.0.0", "id": "0332200e-a7e7-44a8-88b3-a124ea60a5ee", "result": { "score": { "scaled": 0 } }, "verb": { "id": "http://adlnet.gov/expapi/verbs/scored", "display": { "de-DE": "erreichte", "en-US": "scored", "fr-FR": "a marqué", "es-ES": "anotó", "ar-AR": "سحل النقاط" } }, "object": { "id": "http://4198-46917_SCORM_2004 4th Edition", "definition": { "type": "http://adlnet.gov/expapi/activities/lesson" }, "objectType": "Activity" } }, "hasGeneratedId": false, "deadForwardingQueue": [], "voided": false, "verbs": [ "http://adlnet.gov/expapi/verbs/scored" ], "personaIdentifier": "64366f5ef65611193831c0e1", "processingQueues": [], "person": { "_id": "64366f5ec91ba105d0c8c463", "display": "97774 - https://www.courseticket.com (xAPI Account)" }, "__v": 1, "timestamp": "2023-04-12T12:30:48.988Z", "relatedActivities": [ "http://4198-46917_SCORM_2004 4th Edition", "http://4198-46917_SCORM_2004 4th Edition?attemptId=97774_a1", "https://www.courseticket.com/de/user-service-detail/159724/e-learning/4198", "https://w3id.org/xapi/scorm" ], "relatedAgents": [ "https://www.courseticket.com|97774", "mailto:email@example.com" ], "organisation": "60799313462e7205c3a5de1c", "_id": "6436a478697be105e2adbac1", "registrations": [], "pendingForwardingQueue": [] } ```
### H5p **completed** verb
Click to see H5p completed verb ``` { "stored": "2023-07-07T14:23:25.844Z", "active": true, "completedForwardingQueue": [], "failedForwardingLog": [], "client": "6079932d462e7205c3a5de21", "lrs_id": "6079932d462e7205c3a5de20", "completedQueues": [ "STATEMENT_FORWARDING_QUEUE", "STATEMENT_PERSON_QUEUE", "STATEMENT_QUERYBUILDERCACHE_QUEUE" ], "activities": [ "http://1076-51394_H5P_" ], "hash": "7e38d6f1a77df24a94c54fb5282d1b585cafe592", "agents": [ "https://www.courseticket.com|99141" ], "statement": { "authority": { "objectType": "Agent", "name": "leraninglocker-ct", "mbox": "mailto:email@example.com" }, "stored": "2023-07-07T14:23:25.844Z", "context": { "contextActivities": { "category": [ { "id": "http://h5p.org/libraries/H5P.QuestionSet-1.20", "objectType": "Activity" } ] } }, "actor": { "account": { "name": "99141", "homePage": "https://www.courseticket.com" }, "objectType": "Agent", "name": "Gianna" }, "timestamp": "2023-07-07T14:23:25.844Z", "version": "1.0.0", "id": "0a0d0dcc-e811-4d20-a589-871660b73201", "result": { "score": { "min": 0, "max": 30, "raw": 13, "scaled": 0.4333 }, "completion": true, "success": false, "duration": "PT1805.88S" }, "verb": { "id": "http://adlnet.gov/expapi/verbs/completed", "display": { "en-US": "completed" } }, "object": { "id": "http://1076-51394_H5P_", "objectType": "Activity", "definition": { "extensions": { "http://h5p.org/x-api/h5p-local-content-id": "default" }, "name": { "en-US": "Wissensteil (Teil A)_Online Testsim 2023" } } } }, "metadata": { "https://learninglocker&46;net/result-duration": { "seconds": 1805.88 } }, "hasGeneratedId": true, "deadForwardingQueue": [], "voided": false, "verbs": [ "http://adlnet.gov/expapi/verbs/completed" ], "personaIdentifier": "64a818fdf6561146e5344b51", "processingQueues": [], "person": { "_id": "64a818fdc91ba105d0c8c569", "display": "Gianna" }, "__v": 1, "timestamp": "2023-07-07T14:23:25.844Z", "relatedActivities": [ "http://1076-51394_H5P_", "http://h5p.org/libraries/H5P.QuestionSet-1.20" ], "relatedAgents": [ "https://www.courseticket.com|99141", "mailto:email@example.com" ], "organisation": "60799313462e7205c3a5de1c", "_id": "64a81fdd55d8930605c6de1a", "registrations": [], "pendingForwardingQueue": [] } ```
# User behavior tracker ## Abstract The [User Behavior Tracker project](https://gitlab.com/eduplex-api/user-behavior-tracker) aims to offer an alternative to Analytics for tracking user interactions and page activity. By leveraging JavaScript, the system captures user behavior and converts it into **xAPI statements**, enabling the storage of this data in an xAPI-compatible Learning Record Store (LRS), such as [Learning Locker](https://github.com/LearningLocker/learninglocker). This approach allows for the tracking of various learning metrics, providing a powerful and flexible solution for analyzing user engagement and activity. The prototype **successfully demonstrated the ability to capture and store key metrics**, confirming the system's effectiveness. This project provides a practical tool for organizations seeking a robust, privacy-conscious alternative to traditional analytics platforms, with the added benefit of compatibility with xAPI standards. ## Required files Main git repository with source code: * `JsTracking.js` is the main file from this project , which exports all the functions needed to make the tracking work. * `verbs.js` is just a collection of verbs used in the xAPI statements to describe user actions. * `xapiwrapper.min.js` is a script injected automatically in the page head by `JsTracking.js`. This is needed to create the `window.ADL` object, which provide all the functions needed to store xAPI data in the LRS. ## Configuration You will need to set several configuration values in order to communicate with the LRS where the tracking data will be stored: ``` { endpoint: BASE_URL + "edu/api/v1/xAPIProxy/", user: userId, password: userToken, lmsHomePage: "https://proto.eduplex.eu" } ``` We have implemented (in the same backend repository as the Module Occupation Matching) a proxy to forward xAPI requests from our backend to the LRS in order to keep credentials safe. * `userId`: logged-in user ID. * `userToken`: logged-in user access token ## How does it work When you add this project as a package into your application you can access 2 modules: * `jsTrackingFunctions` which contains all the functions you will need to call to track in your platform. Those functions are: * `initXapiTrack` - This function initializes the LRS and other configurations needed to start tracking. It is an asynchronous function, and it is needed to be called before any other tracking function. It receives one parameter, an object like the one in the section above with the lrs configuration. * `sendXapiStatement` - This one triggers a tracking event, which is stored in the LRS as a statement. It receives a parameter with the name of the event, for example: `viewed`, `progressed`, `responded`, `log in`... * `verbsList` - This is a group of constants for the available verbs. For example if you want to trigger a `viewed` event you can use the constant `verbsList.viewed.key` to access the name of the event. Some verbs have a callback function to process some extra data, for example when tracking a `search` event you can include in the xAPI data what the user was looking for and even filters applied to the search. This callback function is optional but each verb will expect different objects in this function. Before sending any tracking event the identifier of the logged-in user must be stored in `window.JsTrackingUserLogged`. Every statement saved in the LRS includes data from the current page. Also, if the page has structured data with format in a `application/ld+json` script all this data is stored inside the statement. ## How to extend and customize statements To modify statements is as simple as editing the file [verbs.js](https://gitlab.com/eduplex-api/user-behavior-tracker/-/blob/main/src/verbs.js). ### How to create new verbs or modify existing ones Verbs have 3 basic properties: * `id`: A unique identifier of the verb (it needs to be a URL). * `key`: Internal name used when you call the `sendXapiStatement` function. * `display`: The name displayed on the LRS. With this in mind you can modify properties of the existing verbs on the file or just add a new one. First it is recommended to check if there is already a verb on the [xAPI registry](https://registry.tincanapi.com/#home/verbs) that fits your case. If you need or want to create your own custom verb you can do it but remember to use always a URL as the `id`. We always use the same URLs under our own domain , but you can use your own. To create a new verb just add one JSON object like this on the verbs file and change it as you like: ``` { id: "https://proto.eduplex.eu/xapi/verbs/custom_verb", display: { "en-US": "Custom verb" }, key: "customVerb" } ``` Once you have added the new verb you can use it on the statements you are sending like this: ``` jsTrackingFunctions.sendXapiStatement(verbsList.customVerb.key) ``` ### How to extend a statement Besides the 3 basic properties of the verb we implemented a new one called `customOptions`. It is a function that allows to customize the `object` of the statement or adding more data to the `extensions` section inside `context`. Params: * You can send any parameter when you do the call of the tracking function `sendXapiStatement` to pass the relevant data for that event. Return: This function is expected to return an object with the following properties: * `object`: This will overwrite the object of the statement. * `extensions`: This will **not** overwrite but add them to the existing ones. They are not mandatory but if you do not need to change any of them you should not include this function and keep the verb simple with the basic properties. Some of the already defined verbs use it. For example, the `searched` verb, with its own `objectType` and one additional `extension`: ``` const searchExtensionKey = 'https://proto.eduplex.eu/xapi/extension/search' const searchObjectType = 'https://proto.eduplex.eu/xapi/object/types/search/' ``` ``` searched: { id: 'http://activitystrea.ms/schema/1.0/search', display: { 'en-US': 'searched' }, key: 'searched', customOptions: (options) => { const searchQuery = options.searchQuery || 'empty_string' const filters = options.filters || {} return { object: { id: searchObjectType + searchQuery, definition: { name: { 'en-Us': searchQuery }, type: searchObjectType }, objectType: 'Activity' }, extensions: { [searchExtensionKey]: { searchQuery, filters } } } } } ``` In the section below you can find an example of how to call the track function with custom options. ## Example In this example we are initializing the tracking and sending a `viewed` event. This is storing an xAPI statement in the LRS for the user `tracking-test-user` with the data from the current page. ``` window.JsTrackingUserLogged = 'tracking-test-user' await jsTrackingFunctions.initXapiTrack({ endpoint: "https://lrs-eu-central-1.eduplex.eu/data/xAPI/", user: "64b9634d139ecf5d77ec8bf335c582c8ee374f5", password: "password1234", lmsHomePage: "https://proto.eduplex.eu" }) jsTrackingFunctions.sendXapiStatement(verbsList.viewed.key) ``` Also, an example of an event with custom options like the `search` mentioned before: ``` const options = { searchQuery, filters } jsTrackingFunctions.sendXapiStatement(verbsList.viewed.key, options) ``` Where `searchQuery` is a string and `filters` is an object. ## Example statement ``` { "authority": { "objectType": "Agent", "name": "EduPLEx Client", "mbox": "mailto:test@eduplex.eu" }, "stored": "2022-09-26T12:06:08.158Z", "context": { "extensions": { "https://proto.eduplex.eu/xapi/extension/device_token": "929f703a-9e42-4c44-897b-735bd1679aa8", "https://proto.eduplex.eu/xapi/extension/course_data": { "hasCourseInstance": { "endDate": "2023-01-16T14:00:00+01:00", "name": "A test course", "startDate": "2022-08-15T01:05:00+02:00", "location": { "@type": "Place", "address": { "@type": "PostalAddress", "addressCountry": { "@type": "Country" } } }, "offers": { "@type": "Offer", "url": "https://www.eduplex.eu/en/e/a-game-with-mushrooms-and-dinosaurs-24373/4989364bc6?c=166811", "name": "A test course", "availability": "InStock", "price": 10, "priceCurrency": "EUR", "validFrom": "2022-06-19T00:00:00+02:00" }, "@type": "CourseInstance", "image": "https://eduplex.imgix.net/e/img/24373-a-game-with-mushrooms-and-dinosaurs-event_pic.jpeg?v=4&auto=compress&w=1920&h=606&fit=crop", "description": "Learn the basics...", "instructor": [], "courseMode": "online" }, "name": "A test course", "timeRequired": "PT0H30M", "url": "https://www.eduplex.eu/en/e/a-game-with-mushrooms-and-dinosaurs-24373/4989364bc6?c=166811", "@context": "http://schema.org/", "provider": { "@id": "https://www.eduplex.eu/es/u/nordev", "@type": "Organization", "name": "Nordev", "url": "https://www.eduplex.eu/es/u/nordev" }, "isAccessibleForFree": "http://schema.org/False", "@type": "Course", "@id": "https://www.eduplex.eu/en/e/a-game-with-mushrooms-and-dinosaurs-24373/4989364bc6?c=166811", "aggregateRating": { "@type": "AggregateRating", "ratingValue": 4.67, "bestRating": 5, "reviewCount": 3 }, "description": "Learn the basics...", "inLanguage": "en" } } }, "actor": { "account": { "homePage": "https://proto.eduplex.eu", "name": "dani-test-tracking" }, "objectType": "Agent" }, "timestamp": "2022-09-26T12:06:08.158Z", "version": "1.0.0", "id": "3a2f0cb5-2ac5-4d17-a2ad-7704785c3d63", "verb": { "id": "http://id.tincanapi.com/verb/viewed", "display": { "en-US": "viewed" } }, "object": { "id": "https://www.eduplex.eu/en/e/a-game-with-mushrooms-and-dinosaurs-24373/4989364bc6?c=166811", "definition": { "type": "http://adlnet.gov/expapi/activities/course" }, "objectType": "Activity" } } ``` ## Other considerations In projects with webpack and babel we found a recurring error with optional chaining operator (`?.`), it shows something like this in build time: `Module parse failed: Unexpected token You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file.` It can be fixed by adding this plugin to your webpack/babel config: `'@babel/plugin-proposal-optional-chaining'` Also make sure webpack transpile our code, in a nuxt project this is as simple as adding the next line inside the `build` property in `nuxt.config.js`. `transpile: ['user-behavior-tracker']` ## License The source code for the site is licensed under the [**MIT license**](https://gitlab.com/eduplex-api), which you can find in the [LICENSE](https://gitlab.com/eduplex-api/user-behavior-tracker/-/blob/main/LICENSE/README.md) file. `xapiwrapper.min.js` file has been taken from another open source repository () which is under [Apache License](http://www.apache.org/licenses/LICENSE-2.0) # List of verbs used * general repository: * current list of verbs available to track (it is easy to add more verbs from existing ones or even create our own verbs): * table of verbs being tracked (continuously extended, a few more already added - work in progress!):
… and a list of actions we are currently tracking in eduplex: * logged in * logged out * search * page view * initiatives score (assessments dashboard) * added to wishlist * reordered an element from wishlist * removed an element from wishlist * purchased * file opened * pdf download # Video wrapper with xAPI tracking ## Required files We need to add [this script](https://gitlab.com/eduplex-api/scorm-to-xapi-wrapper/-/raw/main/src/xapiwrapper.min.js) in the page with the video we want to track using xAPI statements. This will create the window\.ADL object we will use to send xAPI statements to LRS. *** ## Potential Use Case
… track individual/global behaviour inside video (see youtube)! ## Setup Before sending any request, first we need to provide LRS configuration ``` initXAPITrack () { const conf = { endpoint: 'xxxxxxx', user: 'xxxxxxx', password: 'xxxxxxx' } window.ADL?.XAPIWrapper?.changeConfig(conf) }, ``` *** ## Implementation Once we have the LRS configuration ready we can start sending requests to LRS using this function: ``` sendXAPIStatement (action) { const statement = { actor: { account: { homePage: 'xxxxxxx', name: 'xxxxxxx' }, objectType: 'Agent' }, verb: action, object: { id: 'xxxxxxx', objectType: 'Activity' } } window.ADL?.XAPIWrapper?.sendStatement(statement) } ``` The statement structure we are using has 3 parts: * Actor: is the user watching the video * Verb: is the action the user did * Object: identifies the video This function receives one parameter, the action or verb that we send to LRS. You can create your own verbs but its better to use standard ones, which are already defined in `window.ADL.verbs`. They have the following structure, for example for the verb `window.ADL.verbs.initialized`: ``` { "id": "http://adlnet.gov/expapi/verbs/initialized", "display": { "de-DE": "initialisierte", "en-US": "initialized", "fr-FR": "a initialisé", "es-ES": "inicializó", "ar-AR": "بدأ" } } ``` Next step, we have to select which verbs we will use to track the video, in our case we decided to use: * `initialized` when video starts * `suspended` when video pauses * `resumed` when video plays again * `completed` when video ends So now we know which verbs we want to use and we have all the configuration done, we just have to set listenners for each video event and call the function mentioned above. For example: * on video starts → `sendXAPIStatement(window.ADL.verbs.initialized)` * on video pause → `sendXAPIStatement(window.ADL.verbs.suspended)` * on video play → `sendXAPIStatement(window.ADL.verbs.resumed)` * on video end → `sendXAPIStatement(window.ADL.verbs.completed)` You can link any event with any verb you want and this function will send statements to the LRS each time those events are triggered. # E-commerce tracking using xAPI * general repository: [EduPLEx API / User behavior tracker · GitLab](https://gitlab.com/eduplex-api/user-behavior-tracker) Currently we are tracking **purchases.** * **verb**: `http://activitystrea.ms/schema/1.0/purchase` * **extension key:** `https://proto.eduplex.eu/xapi/extension/purchase` * **extension data:** ``` { "ecommerce": { "transaction": { "id": "X-XXXXXXXXXXX", "affiliation": "CT", "revenue": 0.1, "shipping": 0, "price_total": 0.1, "currency": "EUR" }, "items": [ { "id": "E-20", "name": "E-20", "brand": 4, "category": "Event-de", "price": 0, "quantity": 1, "price_unit": 0, "currency": "EUR" } ] } } ``` As every statement we are saving also data from the **offer** in the extension key `https://proto.eduplex.eu/xapi/extension/course_data` ## Example
Click here to watch an example of the statement ``` { "stored": "2023-08-08T13:22:17.764Z", "active": true, "completedForwardingQueue": [], "failedForwardingLog": [], "client": "62ed2d33caa1070008fbe17d", "lrs_id": "62ed2d33caa1070008fbe17c", "completedQueues": [], "activities": [ "https://proto.eduplex.eu/xapi/object/types/purchase/4-1691500934526" ], "hash": "4dc2bbe650d6ee76575038b1eb23b91edfda9581", "agents": [ "https://proto.eduplex.eu|3" ], "statement": { "authority": { "objectType": "Agent", "name": "EduPLEx Client", "mbox": "mailto:adrian@courseticket.com" }, "stored": "2023-08-08T13:22:17.764Z", "context": { "extensions": { "https://proto.eduplex.eu/xapi/extension/device_token": "d95906a5-aec8-4234-9092-025c5c5b227d", "https://proto.eduplex.eu/xapi/extension/course_data": { "additionalType": 1, "coursePrerequisites": [], "teaches": [ "Public test module", "Test module", "Text module", "h5p berries", "question setdddd", "xapi version test", "scorm versionf", "xapi hack", "xapi module partial content test", "Video partial test " ], "learningResourceType": 12, "hasCourseInstance": [ { "@type": "CourseInstance", "offers": { "validFrom": "2023-08-08T13:22:09.663Z", "itemOffered": { "@type": "Product", "additionalProperty": [ { "@type": "PropertyValue", "name": "is_internal", "value": false }, { "@type": "PropertyValue", "name": "vat_effective", "value": 0 }, { "@type": "PropertyValue", "name": "provider_is_topseller", "value": false }, { "@type": "PropertyValue", "name": "provider_is_verified", "value": true } ] }, "price": 0, "includesObject": { "@type": "TypeAndQuantityNode", "amountOfThisGood": 10, "unitText": "modules" }, "name": "Test1", "url": "https://proto.eduplex.eu/de/e/test1-20", "priceCurrency": "EUR", "priceSpecification": { "@type": "PriceSpecification", "maxPrice": 0, "minPrice": 0, "valueAddedTaxIncluded": "http://schema.org/True" }, "@type": "Offer", "availability": "http://schema.org/InStock" }, "instructor": [], "courseMode": "blended", "startDate": "2023-08-08T13:22:09.663Z", "endDate": "2024-08-08T13:22:09.662Z" } ], "keywords": [ "Web-Designer/Web-Designerin", "mithilfe digitaler Technologien zusammenarbeiten", "IT-Projektmanager/IT-Projektmanagerin ", "Leiterplatten testen", "3D-Druckvorgang", "2D-CAD-Technik bei der Erstellung von Mustern für Schuhwerk einsetzen", "Prüfverfahren zur Tests der Wasserqualität durchführen", "Textiltestverfahren durchführen", "2D-Pläne lesen", "chemische Laboruntersuchungen an Metallen vornehmen" ], "name": "Test1", "timeRequired": "PT4H20M", "url": "https://proto.eduplex.eu/de/e/test1-20", "@context": "http://schema.org/", "educationalAlignment": [ { "@type": "AlignmentObject", "alignmentType": "occupation", "targetUrl": "http://data.europa.eu/esco/occupation/c40a2919-48a9-40ea-b506-1f34f693496d" }, { "@type": "AlignmentObject", "alignmentType": "skill", "targetUrl": "http://data.europa.eu/esco/skill/2b34a99f-9813-4c91-9509-b6b9b8c3132e" }, { "@type": "AlignmentObject", "alignmentType": "occupation", "targetUrl": "http://data.europa.eu/esco/occupation/8b6388a4-4904-471b-9331-d3b1211f5525" }, { "@type": "AlignmentObject", "alignmentType": "skill", "targetUrl": "http://data.europa.eu/esco/skill/d1f435d2-e574-482c-8187-4a6c2214c683" }, { "@type": "AlignmentObject", "alignmentType": "skill", "targetUrl": "http://data.europa.eu/esco/skill/2afb2b59-c9a3-4cf3-b1dd-1a2fad51e583" }, { "@type": "AlignmentObject", "alignmentType": "skill", "targetUrl": "http://data.europa.eu/esco/skill/73c7d0d9-092f-4f6e-800a-5759f45a6fc6" }, { "@type": "AlignmentObject", "alignmentType": "skill", "targetUrl": "http://data.europa.eu/esco/skill/e80f9e50-71ff-4bb7-8b7d-eb232685d17f" }, { "@type": "AlignmentObject", "alignmentType": "skill", "targetUrl": "http://data.europa.eu/esco/skill/eb13f8c5-8300-4422-a51f-475a7edb09ef" }, { "@type": "AlignmentObject", "alignmentType": "skill", "targetUrl": "http://data.europa.eu/esco/skill/dfce2aba-783e-4e17-9c17-e74aca505ec2" }, { "@type": "AlignmentObject", "alignmentType": "skill", "targetUrl": "http://data.europa.eu/esco/skill/2b60c0cf-6ce6-4f04-9748-0e6d883673d8" } ], "provider": { "@id": "https://proto.eduplex.eu/de/u/demo", "@type": "Organization", "name": "demo", "url": "https://proto.eduplex.eu/de/u/demo", "image": "https://edupl.imgix.net/u/img/4-profile_pic.png?v=4&auto=compress&w=700&h=700&fit=crop" }, "isAccessibleForFree": "http://schema.org/True", "@type": "Course", "image": "https://edupl.imgix.net/e/img/20-test1-event_pic.jpeg?v=2&auto=compress&w=1920&h=606&fit=crop", "@id": "https://proto.eduplex.eu/de/e/test1-20", "description": "Only a testing......", "inLanguage": "de" }, "https://proto.eduplex.eu/xapi/extension/purchase": { "ecommerce": { "transaction": { "id": "4-1691500934526", "affiliation": "CT", "revenue": 0.1, "shipping": 0, "price_total": 0.1, "currency": "EUR" }, "items": [ { "id": "E-20", "name": "E-20", "brand": 4, "category": "Event-de", "price": 0, "quantity": 1, "price_unit": 0, "currency": "EUR" } ] } } } }, "actor": { "account": { "homePage": "https://proto.eduplex.eu", "name": "3" }, "objectType": "Agent" }, "timestamp": "2023-08-08T13:22:17.764Z", "version": "1.0.0", "id": "2630b7af-62a4-464a-909f-9197548532bc", "verb": { "id": "http://activitystrea.ms/schema/1.0/purchase", "display": { "en-US": "purchased" } }, "object": { "id": "https://proto.eduplex.eu/xapi/object/types/purchase/4-1691500934526", "definition": { "name": { "en-Us": "E-20" }, "type": "https://proto.eduplex.eu/xapi/object/types/purchase/" }, "objectType": "Activity" } }, "hasGeneratedId": false, "deadForwardingQueue": [], "voided": false, "verbs": [ "http://activitystrea.ms/schema/1.0/purchase" ], "processingQueues": [], "person": null, "timestamp": "2023-08-08T13:22:17.764Z", "relatedActivities": [ "https://proto.eduplex.eu/xapi/object/types/purchase/4-1691500934526" ], "relatedAgents": [ "https://proto.eduplex.eu|3", "mailto:adrian@courseticket.com" ], "organisation": "62ed1d89858c0c0008338e5c", "_id": "64d24189e13da3001b310384", "registrations": [], "pendingForwardingQueue": [] } ```
# Playlist builder UI The [Playlist Builder UI project](https://gitlab.com/eduplex-api/nuxt-listing) focuses on providing a flexible and efficient solution for **displaying and managing** playlists and **lists of learning content**. **Designed as a library**, it offers a collection of versatile **components** that simplify the process of building and customizing lists. This tool enhances user experience by enabling content providers and developers to create intuitive and organized interfaces for presenting learning materials. The Playlist Builder UI serves as a practical resource for seamlessly integrating list and playlist functionalities into educational platforms. *** **Nuxt-Listing** repository (). Display and manage any type of list and playlists with learning content. This library provides a set of handy components to build list. ## License The source code for the site is licensed under the [**MIT license**](https://gitlab.com/eduplex-api), which you can find in the [LICENSE](https://gitlab.com/eduplex-api/scorm-to-xapi-wrapper/-/blob/main/LICENSE) file. # Technical documentation ## Components ### ListingLists #### Description This component renders the view of the application, allows to choose any list & displays the data of the current list. Usage example: ``` listing-lists( :bookmark-url="https://www.example.com/e/1?m=1&slide=#slide1" :http="axios" :shared-url="https://www.example.com/list" :stored-list="[{}]" :open-list-id="1" :user-id="3" @uppdate-list="updateList" @delete-event="refreshEventsInLists" @navigate-to-bookmark="navigateToBookmark" ) ``` #### Props | Name | Type | Required | Description | | ------------ | -------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | | bookmark-url | String | false | URL of current slide in the context of a XAPI module | | http | Function | true | Promise-based HTTP Client including logic to add credentials (axios is recommended). This will be used in `API.service.js` to perform requests | | shared-url | String | true | URL of the page to render the public list in the library component `listing-shared-list` | | stored-list | Array | false | List of objects corresponding to each list | | open-list-id | Number | false | Id of the list to be opened | | user-id | Number | true | Id of the current user | #### Events @update-list (Emits a flag when the item was successfully added) @delete-event (Emits a flag when any wish list or event added in any wish list was removed) @navigate-to-bookmark (Emits the saved note url when the chip button is clicked) #### HTTP Requests It uses ***GET*** request to fetch the lists of the current user. It uses ***GET*** request to fetch the notes of the current list. It uses ***POST*** request to create lists. It uses ***PATCH*** request to edit lists. It uses ***DELETE*** request to remove lists. ### ListingShared #### Description This component allows to render any public list as read only in a custom page. Usage example: ``` listing-shared(:http="axios" :slug="XXXXXXXXX") ``` #### Props | Name | Type | Required | Description | | ---- | -------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | | http | Function | true | Promise-based HTTP Client including logic to add credentials (axios is recommended). This will be used in `API.service.js` to perform requests | | slug | String | false | ID of the selected public list that is send in the parameters in a HTTP request to load a public list | #### Events (This component does not have any emit) #### HTTP Requests It uses ***GET*** request to fetch the items from a public list. ### ListingAddItemList #### Description This component allows to render a selector that allow to add the current event into a list. Usage example: ``` listing-add-item-list( :fetching-list="false" :http="axios" :refresh-events-in-lists="getRefreshEventsInLists" :list-item="course" :stored-list="[{}]" :user-id="3 @item-added="newItemAdded" @update-list="updateList" @fetching-list="fetchList" @refresh-lists="refreshEventsInLists" ) ``` #### Props | Name | Type | Required | Description | | ----------------------- | -------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | | fetching-list | Boolean | false | Flag to control the `GET request to fetch the list`, if its set to **true**, then the request its omitted and load the results from prop `:stored-list` | | http | Function | true | Promise-based HTTP Client including logic to add credentials (axios is recommended). This will be used in `API.service.js` to perform requests | | refresh-events-in-lists | Boolean | false | Flag to notify the application to trigger the `GET request to fetch if the current event itd added in any list` | | list-item | Object | true | Object data of the current event. It must be mapped with the properties: "title", "image" & "url" | | stored-list | Array | false | Set the internal list to the value of this prop | | user-id | Number | true | Id of the current user | #### Events @item-added (Emits a flag when the item was successfully added) @update-list (Emits the response of the `GET list request`) @fetching-list (Emits a flag to handle the `GET list request` to don't send the same request multiple times) @refresh-lists (Emits a flag to notify the app to handle the `GET list request with url parameter` to fetch if the current event itd added in any list) #### HTTP Requests It uses ***GET*** request to fetch the items from a list. It uses ***GET*** request to fetch the lists. It uses ***GET*** request to fetch if the current event is already added in any list. It uses ***POST*** request to add an item to a lists. ### ListingAddItemListIconDesign #### Description This component allows to render a button that allow to add the current event into the **first** list. It extends from the component `ListingAddItemList` Usage example: ``` listing-add-item-list-icon-design( add-icon="mdi-heart-plus" already-added-icon="mdi-heart-multiple" :fetching-list="false" :http="axios" icon-color="#ff0000" :icon-size="30" :refresh-events-in-lists="getRefreshEventsInLists" :list-item="course" :stored-list="[{}]" :user-id="3 @item-added="newItemAdded" @update-list="updateList" @fetching-list="fetchList" @refresh-lists="refreshEventsInLists" ) ``` #### Props | Name | Type | Required | Description | | ----------------------- | -------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | | add-icon | String | false | Icon to display while the current event its not stored in the wish list | | already-added-icon | String | false | Icon to display while the current event its stored in any wish list | | fetching-list | Boolean | false | Flag to control the `GET request to fetch the list`, if its set to **true**, then the request its omitted and load the results from prop `:stored-list` | | http | Function | true | Promise-based HTTP Client including logic to add credentials (axios is recommended). This will be used in `API.service.js` to perform requests | | icon-color | String | false | Color of the icons to add an event | | icon-size | Number | false | Size of the icons | | refresh-events-in-lists | Boolean | false | Flag to notify the application to trigger the `GET request to fetch if the current event itd added in any list` | | list-item | Object | true | Object data of the current event. It must be mapped with the properties: "title", "image" & "url" | | stored-list | Array | false | Set the internal list to the value of this prop | | user-id | Number | true | Id of the current user | #### Events @item-added (Emits a flag when the item was successfully added) @update-list (Emits the response of the `GET list request`) @fetching-list (Emits a flag to handle the `GET list request` to don't send the same request multiple times) @refresh-lists (Emits a flag to notify the app to handle the `GET list request with url parameter` to fetch if the current event itd added in any list) #### HTTP Requests It uses ***GET*** request to fetch the items from a list. It uses ***GET*** request to fetch the lists. It uses ***GET*** request to fetch the events from the **first** list. It uses ***GET*** request to fetch if the current event is already added in any list. It uses ***POST*** request to add an item to a lists. ### Other internal components | Name | Description | | -------------------- | ------------------------------------------------------------------------------------------------------------- | | `AddNewListBtn` | Display a button to create new lists. Emits the newly created list as `@new-list`. | | `CreateNewListModal` | Display a modal to create new lists. Emits `@update:displayModal` & `@new-list` to `newListBtn` component. | | `OptionsBtn` | Display a button and a menu with some options and emits the selected option. | | `DeleteModal` | Display a modal to remove the current list or note. Emits `close` & `@delete` | | `EditModal` | Display a modal to edit the current list or a note. Emits `close` & `@edit` | | `ListWrapper` | Provide common variables & functions that will be inherited to `NotesList` or `WishList`. Emits `@edit-error` | | `NotesList` | Display the notes of the current selected list. | | `WishList` | Display the events of the current selected list. | ## Internationalization Nuxt I18n is used. Some files already translated are located in the directory `lib/i18n`. Those files can be directly imported in the base project using `i18n.mergeLocaleMessage`: ``` this.$i18n.mergeLocaleMessage(this.$i18n.locale, require('../../lib/i18n/' + this.$i18n.locale + '/views')) ``` ## Prerequisites This library uses the following dependencies: * [Pugjs](https://pugjs.org/api/getting-started.html) instead of plain HTML for templating * [Vuetifyjs](https://vuetifyjs.com/en/) As UI design framework based on Material * [NuxtJS](https://nuxtjs.org/) 2 (use in Vuejs or Nuxt 3 on your own risk) * [Lodash](https://lodash.com/) Library which provide functions for working with arrays, numbers, objects, strings * [MomentJs](https://momentjs.com/) Library to parse, validate, manipulate, and display date/time * [VueSlickSort](https://www.npmjs.com/package/vue-slicksort) Library to perform drag and drop of items in a list ## Development * Clone the repository `git clone https://gitlab.com/eduplex-api/nuxt-listing.git` in the root of any Nuxtjs app. * Import statically components manually when needed using `import { ListingLists } from '../../nuxt-listing/index'` * Import dynamically components manually using `ListingShared: () => import('../../nuxt-listing/shared')` * Build your project as usual # SSI/VC Edu Wallet Demo Application ## Abstract The SSI/VC EduWallets project introduces a proof-of-concept demo application for issuers and verifiers, enabling seamless integration of Self-Sovereign Identity (SSI) functionalities into third-party platforms like Learning Management Systems (LMS) or learning platforms. The solution utilizes European Digital Identity (EUDI) wallets to issue and verify educational Verifiable Credentials (VCs), providing a secure, decentralized, and privacy-focused approach to managing user data and credentials in the Web 3.0 paradigm. By leveraging standards such as W3C Decentralized Identifiers (DIDs), Verifiable Credentials Data Model, and the ESCO taxonomy, SSI EduWallets ensure interoperability, cryptographic security, and detailed accreditation of skills and qualifications. The platform enables efficient issuance of tamper-proof digital diplomas and streamlines credential verification, enhancing learning experiences and reducing time and costs associated with traditional methods. Targeted at platform operators and providers, SSI EduWallets prioritize privacy, security, and user control, addressing challenges in data handling, interoperability, and trust in credential systems. *** Back-end and front-end for the issuer and verifier demo application. This source code it is supposed to be a demo that can serve as implementation example. It should be implemented as part of a 3rd party application (LMS, learning platform, etc) in order to take advantage of SSI Edu Wallets. ## License The source code for the site is licensed under the MIT license, which you can find in the LICENSE file. # Introduction The Self-Sovereign Identity (SSI) EduWallets project aims to revolutionize the way we manage our personal information and privacy in the digital age of the Web 3.0. By harnessing the power of decentralized and distributed technologies such as blockchain, verifiable credentials (VCs) and self-sovereign identity, this project seeks to give users complete control over their data, freeing it from the silos of third-party storage and therefore build a more transparent and trustworthy digital ecosystem.
[*Different data paradigms along the web generations.*](#user-content-fn-1)[^1] The SSI EduWallets project focuses on the implementation of a user management system for online platforms. Users are allowed to interact with the platforms that implement this system and perform the issuance of educational verifiable credentials once they complete a course or assessment through the use of [European digital identity (EUDI)](#user-content-fn-2)[^2] wallets. The presentation of verifiable credentials that are stored in the user’s wallet help to enhance the platform’s “learning experiences” by verifying learning histories and using this data to provide a better overall experience on the platform. This approach, based on SSI, [W3C Decentralized identifiers (DIDs)](#user-content-fn-3)[^3],[ W3C Verifiable Credentials Data Model (VCs)](#user-content-fn-4)[^4] and EUDI/ [European Self-Sovereign Identity Framework (ESSIF)](#user-content-fn-5)[^5] standards, ensures interoperability and security between different systems within the European Union. Once a user receives a verifiable credential it will be stored in the user’s wallet for future verification. By digitally signing each verifiable credential and ensuring standardization of formats, the project seeks to end the problems of interoperability, forgery and inconsistency that plague traditional educational certificate systems. [^1]: European Commission The European Digital Identity Wallet Architecture and Reference Framework, Shaping Europe’s digital future. Available at: [https://digital-strategy.ec.europa.eu/en/library/european-digital-identity-wallet-architecture-and-reference-framework](/ssi-edu-wallets/introduction) (Accessed: 08 August 2023). [^2]: European Commission The European Digital Identity Wallet Architecture and Reference Framework, Shaping Europe’s digital future. Available at: (Accessed: 08 August 2023 [^3]: Sporny, M. et al. Decentralized identifiers (DIDs) v1.0, W3C. Available at: (Accessed: 08 August 2023). [^4]: Sporny , M., Longley , D. and Chadwick , D. Verifiable credentials data model V1.1, W3C. Available at: (Accessed: 08 August 2023). [^5]: Pastor Matut, C. and Du Seuil, D. Understanding the European self-sovereign identity framework (ESSIF), PPT. Available at: [https://www.slideshare.net/SSIMeetup/understanding-the-european-selfsovereign-identity-framework-essif ](https://www.slideshare.net/SSIMeetup/understanding-the-european-selfsovereign-identity-framework-essif)(Accessed: 08 August 2023). # Project description ## Goals and benefits of the project The main goal of this project is the implementation of a “proof-of-concept” capable of being integrated in educational platforms so third parties can make profit of the new Web 3.0/SSI paradigm and leverage the features of user privacy and the issuance of educational verifiable credentials to the users. Verifiable credentials are a representation of an accreditation of knowledge achieved under a qualification framework like the[ Qualification Metadata Schemata (QMS)](#user-content-fn-1)[^1]. This schemata are used to classify the learning outcomes in a detailed specification and also make use of the [European Skills, Competences, Qualifications and Occupations (ESCO) ](#user-content-fn-2)[^2]taxonomy. Once a user finishes a course or assessment he/she can share verifiable presentations (VPs) for verification and validation of learning achievements via wallet and a platform. The main benefits of the implementation are the capabilities of a platform to issue a digital, cryptographically secure, semantically rich credential that specifies qualifications, skills and occupations under a common framework and, furthermore, to enhance interoperability between independent systems. Certificates which are issued as educational verifiable credentials are easy to verify and difficult to manipulate. The SSI EduWallets implementation seeks to eliminate the problems of the user data handling, the forgery of traditional credentials, and the non-interoperability between systems. It streamlines the process of the issuance of a secure educational credential and the verification of the credentials saving time and costs. ## Target group The SSI EduWallets project is intended for platform operators (SMEs, corporates, governmental, non-governmental) or platform providers (LMS reseller, Learning App provider) which want to extend their platforms with the new paradigm of SSI, eliminating data privacy & security issues related to the management of users’ personal/private data. Once integrated, they allow their users to perform secure identification using EUDI/ESSIF compliant wallets via verifiable presentations (VPs) of learning achievements or issuance of educational verifiable credentials to streamline accreditations & certificates (and leverage the use of DIDs and VCs). ## Content overview SI EduWallets provide a series of software components that create a stack needed to implement the new paradigm of SSI and the use of educational verifiable credentials within learning platforms. Through the components of **EUDI wallets**, **issuer API**, **verifier API,** **wallet kit API**, **issuance and verifier user interface** the implementation contains following user journeys: * Users can use EUDI wallet applications within the platforms that implement SSI EduWallets to exchange data. Those wallets store and manage the users’ data instead of third parties, **enhancing the users’ privacy**. * Platforms that implement SSI EduWallets can **issue digital, cryptographically secure education diplomas** called "educational verifiable credentials" that represent an accreditation of the achievements that a user receives once he/she completes a course or assessment. These verifiable credentials will be stored and managed inside the user's wallets. * The educational verifiable credentials issued consist of the data of the user (holder), the data of the platform (issuer), the data of the course or assessment and the issuer's signature. The information about the course or assessment follows the Qualification Metadata Schemata and ESCO in order to provide **useful information about the skills, qualifications and experiences** that the user achieves through a learning experience in order to **obtain new learning opportunities and occupations**. * Platforms implementing SSI EduWallets can receive verifiable presentations from users' wallets to **verify and validate shared verifiable credential data** and use within platforms to **streamline some internal processes**. * The verifiable credentials are **interoperable between different systems** because they are defined under a common data schema. Independent/external systems can recognize and then parse the structure and constraints of the specific type of verifiable credentials. * The management processes of the interoperability, issuance and verification of the credentials are **more simple and faster improving the user experience, saving costs and time** because the intermediary parties usually in charge of performing those tasks are removed. * Verifiable credentials are **secure because they use asymmetric cryptography technologies and digital signatures** to make them tamper-proof, easy to verify and difficult to forge. * The data of the issuers like DIDs, public keys and transactions are **redundant, highly available and secure** if the system uses the blockchain infrastructure of [European Blockchain Services Infrastructure (EBSI)](#user-content-fn-3)[^3] in order to verify a verifiable credential. [^1]: European Commission (2020) ‘Publishing of Qualification and Learning Opportunity Data Documentation. Available at: [^2]: ESCO About Esco, ESCO. Edited by European Commission. Available at: (Accessed: 08 August 2023). [^3]: European Commission European Blockchain Services Infrastructure, Home - EBSI -. Available at: [https://ec.europa.eu/digital-building-blocks/wikis/display/EBSI/Home ](https://ec.europa.eu/digital-building-blocks/wikis/display/EBSI/Home)(Accessed: 09 August 2023). # Research and challenges ## The problem of today’s user data management Nowadays the society is more concerned about the user data privacy, this is reflected in Europe for example with the [General Data Protection Regulation (GDPR)](#user-content-fn-1)[^1]. Currently there exist enterprises that are handling our data to perform operations like bespoke Identity providers (IDPs). As customers we have to fully trust that these companies do not use our data negligently or illegally for other purposes but in the end we have no proof where the data is going. The new paradigm of Web 3.0 and SSI tries to get rid of those challenges. Now it is the user who has his/her personal data stored in a wallet application and it is the user who chooses which data he/she wants to share and with whom. ## The problem with current credentials Nowadays the credentials pass through complex and slow processes that involves more than one entity to perform actions like the issuance of a educational diploma or the verification of the presentation of a document that accredit something and even can't ensure that the credentials are not forged. They are easy to manipulate or fake and authenticity is very difficult to prove. Another common problem is the poor interoperability that those traditional diplomas have, because they don't usually follow any standard to convey the relevant information in the same way. A simple example: If a user studied in Spain and achieved a degree certificate. He wants to apply for a master degree at a University in Germany. The student and the University administration need to perform some slow processes to validate the degree certificate and then accept the application for the master degree. This problem is solved with SSI EduWallets: Users can receive an educational diploma in the shape of a digital diploma that follows a common standard schemata. This digital diploma is called "verifiable credential", delivered in the format of JSON-LD or JWT, and build upon the information about the student, the platform that issues the verifiable credential, the course or assessment information, and the signature (DID) of the issuer of the verifiable credential (which makes the verifiable credential secure thought the use of asymmetric cryptography). ## Why to use SSI EUDI Wallets on learning platforms? This new paradigm about people's identity on the Web will solve many problems: 1. **Privacy:** SSI wallets allow users to share only the necessary information to verify a credential without revealing unnecessary personal information. This protects the user's privacy and prevents third parties from collecting and using their data without their consent. Over time, the European Union has been implementing regulations regarding what can or cannot be done with user data like the GDPR. With this new paradigm we will face less GDPR breaches in the future. 2. **Interoperability:** SSI wallets support interoperability between different credential issuers and verifiers. This means that users can store all their credentials in one place and share them with different parties without having to create a new identity for each party. 3. **Decentralized and secure:** SSI wallets usually use decentralized and secure technologies, such as blockchain, to store and manage identity information. This means that the user has complete control over his/her personal data and can choose to share it only with trusted parties and without the need for intermediaries or centralized databases. 4. **Trust and security:** SSI wallets use digital signatures and cryptographic techniques to ensure the authenticity and integrity of the credentials. This provides a high level of trust and security for the credential issuer and the verifier. 5. **Portable and convenient:** SSI wallets provide a portable and convenient way to manage and use credentials. Users can access their credentials from any device with an internet connection making it easy to share them as soon as needed. [^1]: European commission (2022) Official Legal Text, General Data Protection Regulation (GDPR). Available at: (Accessed: 12 August 2023). # System architecture ## Technologies and concepts being used ### Self-Sovereign Identity (SSI) SSI is a digital identity concept that gives individuals **control over the information they use to identify themselves** to websites by managing their identity data directly, **without the need for intermediaries or central authorities**, services, and applications on the Internet. In traditional identity systems, individuals often have limited control over their personal data, which is stored and managed by central authorities such as governments, companies, or service providers. This can lead to issues of privacy, security, and lack of user autonomy. SSI is based on **decentralized and cryptographic technologies**, providing individuals with the ability to **share** only the specific information required for a particular transaction or interaction, without disclosing unnecessary **personal data**. Through the SSI EduWallets implementation it is possible to leverage the features of this new paradigm and provide a new way to **exchange data in a secure, agile and interoperable way.** ### European Digital Identity wallets (EUDI) EUDI wallets are applications that allow you to manage the digital identity of a citizen of a country of the European Union, this applications are part of the[ electronic IDentification, Authentication and trust Services 2 (eIDAS 2.0)](#user-content-fn-1)[^1] proposal which provides a single **european digital identity** and allows to perform transactions both at a public and private level with greater security and control of information. EUDI wallets enable users to securely store and access identification data derived from their national eIDs within an app, on a local or remote basis upon request, with full control over their data moreover these wallets allow to store not only verifiable credentials for ID also other verifiable credentials like **education verifiable credentials**. EUDI wallets comply with the **ESSIF** and use the services of **EBSI** to store some relevant information about the identity and transactions in the ledger to ensure data accuracy and security. These wallets will be used by both government services and private companies to centralize multiple procedures such as payments, travel, management of personal identity credentials or the management of academic degrees within the same application. Since there is currently no digital identity provider under eIDAS 2.0 proposal, it is not possible to obtain verifiable identity credentials at the moment, so these credentials cannot currently be used in the implementation of the project. Therefore, the use of these wallets within the platforms implementing the SSI EduWallets is dependent on the use of the DIDs as unique identifiers for each user. ### European Self-Sovereign Identity Framework (ESSIF) ESSIF is an initiative and framework developed by the European Commission to promote and support the **implementation of self-sovereign identity** (SSI) solutions across Europe. It seeks to build a **more user-centric and privacy-preserving approach to digital identity**, supporting the broader goal of a **trusted and secure digital society** in Europe. The goal of ESSIF is to enable individuals to have greater control over their **digital identity** and personal data while ensuring **privacy, security, and interoperability** establishing a standardized and interoperable ecosystem for SSI solutions. ESSIF allows individuals to manage their personal data, decide who can access it, and provide verifiable credentials without the need for intermediaries or central authorities. SSI EduWallets leverage this framework to perform all the activities related to SSI and the management of verifiable credentials. ### Decentralized Identifiers (DID) Decentralized identifiers (DIDs) are a new type of identifier that enables verifiable, **decentralized digital identifiers**. “A DID refers to any subject (e.g., a person, organization, thing, data model, abstract entity, etc)”.
[Decentralized Identifier parts](#user-content-fn-2)[^2] Decentralised Identifiers (DIDs) are the cornerstone of self-sovereign identity (SSI). DIDs are **URL-based identifiers** associated with an entity; “a DID is just a long string that does not provide any meaningful information about a natural or legal entity. DIDs and DID Documents are generated by their owners with their wallet or back-office systems”[. ](#user-content-fn-3)[^3]These identifiers are most often used in a **verifiable credential** and they are associated with subjects such that a verifiable credential itself can be easily ported from one repository to another without the need to reissue the credential. A decentralized identifier document (DID document) is a document that contains information related to a specific decentralized identifier, such as the associated repository and **public-key information** . DIDs are used to **ensure the authenticity of issuers and holders** in machine verifiable documents known as Verifiable Credentials (VCs) Within a decentralized blockchain infrastructure like EBSI, DIDs and DID Documents are aimed at Legal Entities. They are registered in DID Registries that help establish a trust framework and a secure and reliable Decentralised Public Key Infrastructure (DPKI). DID registries enable DID controllers to register, update or deactivate their DIDs and DID Documents. DIDs consist of the schema or "did:", the first part of the definition of a DID, the "method", a mechanism or protocol for creating and managing unique and decentralized identifiers, and the "DID method specific identifier", a completely unique random number that follows method-specific generation rules. SSI EduWallets use DIDs in both “issuer” and “holder”: In the case of the ”issuer” within EBSI, the DID and the public key are stored on the **blockchain** to be verified by third parties. In the case of the “holder”, the DID is generated and stored by the user’s wallet. In case the DIDs are created with the method “key”, then none of them are stored on the blockchain. This method allows **to work with DIDs and VCs without any decentralized service (no blockchain needed!)**. ### Verifiable Credentials Data Model (VC) Credentials are a part of our daily lives; driver's licenses are used to assert that we are capable of operating a motor vehicle, university degrees can be used to assert our level of education, and government-issued passports enable us to travel between countries. Verifiable credentials with DIDs are the core of SSI paradigm, the verifiable credentials provides a **mechanism to express these sorts of credentials digitally on the Web** normally in the format of **JSON-LD or JWT** in a way that is **cryptographically secure, privacy respecting, and machine-verifiable**. The verifiable credentials have features such as: * **Portability**: due they are digital documents, they can be safely shared with third parties. * **Interoperability**: because they are created in the same formats like JSON-LD or JWT and the definition is based on a schema that can be a public standard. * **Security**: VCs use asymmetric encryption and signatures to keep the credentials safe and tamper-proof. * **Selective disclosure**: the individual has control over which pieces of information they share in a given context. They can selectively disclose specific attributes without revealing unnecessary personal data. * **Revocable:** verifiable credentials can be revoked by the issuer if the information they contain becomes outdated or invalid. This adds an additional layer of trust and assurance. In the SSI EduWallets implementation it is used as an **educational verifiable credential**; this is a type of digital credential that represents an **individual's educational achievements**, qualifications, or accomplishments in a verifiable and tamper-resistant manner. It serves as a secure and portable proof of the person's educational history and can be shared with third parties. ### Verifiable Credential schemas [Verifiable credentials schemas](#user-content-fn-4)[^4] are a standardized format or **data model** normally in a JSON-LD format used to **describe**, **define the constraints and structure of any verifiable credential type**. Verifiable credential schemas allow verifiable credentials to be easily shared and verified between different parties because following a standard for defining a verifiable credential **makes them interoperable** between different systems. Within EBSI or other decentralized systems the creation of these schemas usually are published in a public repository to be used and known for all parties within the system. SSI EduWallets define their own schema for the **user learning outcome verifiable credential** that was created, this schema is **based on the**[ **EBSI verifiable diploma schema**](#user-content-fn-5)[^5] in a simplified way, because none of the current definitions are standardized yet. So interoperability will be possible once the issuer is onboard on the EBSI ecosystem and the schema is published. ### Verifiable Presentation (VP) Verifiable presentations are **mechanisms** **like the verifiable credentials** (they are based on JSON-LD or JWT as well) in which the definition about the presentation, the **verifiable credentials** that will be verified, and the **sign and encryption** of the user will be stored. Verifiable presentations have the mission to **demonstrate the validity of their verifiable credentials without revealing unnecessary or sensitive information**. It is a core concept in decentralized identity systems and plays a crucial role in enabling **selective disclosure of information** during identity verification processes. When an individual needs to prove certain attributes or qualifications to a verifier they can create a verifiable presentation. This presentation **acts as a wrapper around one or more verifiable credentials,** allowing the individual to disclose only the specific attributes required for the verification process, **while keeping the rest of the credential data private**. The verifiable presentation includes a cryptographic proof that proves the authenticity and validity of the underlying verifiable credentials without exposing the raw data contained within them. When the verifier tries to verify the verifiable presentation, first it **validates the sign of the user** which is part of the verifiable presentation and then - for each verifiable credential within the verifiable presentation- it validates the **sign of the issuer** within the verifiable credentials. The SSI EduWallets use verifiable presentations to **exchange verifiable credentials** from the user’s wallets to the third parties. The verification of the verifiable credentials is out of the blockchain due the fact that EBSI is not publicly available yet. ### European Blockchain Services Infrastructure (EBSI) EBSI is an initiative of the European Union (EU) aimed at developing a **service infrastructure** based on **blockchain technology** to enhance and strengthen the delivery of public services throughout Europe. EBSI aims to leverage the potential of blockchain technology to provide more **secure, transparent, efficient, and reliable digital services** in various areas and sectors, such as public administration, healthcare, **education**, transportation, **digital identity**, and more. The infrastructure is designed to be **interoperable** and available for use by different EU member states, allowing for greater collaboration and coordination in the provision of digital services at the European level. “The basic architecture of EBSI is composed of three main elements: **APIs**, exposed on the public internet, which allow applications to connect; **Smart Contracts**, which act as a go-between the outside world (APIs) and the **ledger**; that is a **decentralized database** of information that can be accessed by actors looking to complete a business process”. “All of EBSI's Core Technical Services - APIs, Smart Contracts, and the EBSI ledger - are hosted in a decentralized way, by a network of nodes all across Europe. They synchronize their copies of the ledger, making it **distributed**, and all make EBSI's Core Technical Services available”. Basically the EBSI infrastructure serves to **store records such as the DIDs of legal entities, record the transactions that are carried out and verify data by accessing the ledger**. This way the security and the availability is enhanced. The key points of using EBSI are: * **Security and trust**: Blockchain technology guarantees the security and authenticity of data, generating greater confidence in records and transactions. * **Transparency and traceability**: The immutability of the blockchain allows greater transparency and monitoring of operations and data. * **Efficiency and cost reduction**: Process automation improves efficiency and lowers operating costs. * **Interoperability**: EBSI is designed to be used by different countries of the European Union, facilitating collaboration and information exchange. * **Authentication and identity security**: Enables more secure and decentralized digital identity systems. * **Document verification and authenticity**: Guarantees the authenticity of important documents, reducing the risk of fraud. * **Improvement of public services**: Provides a more reliable and secure infrastructure for the provision of digital services. * **Promotion of innovation**: Promotes the development of advanced and efficient digital solutions. SSI EduWallets seeks to leverage the EBSI ecosystem to perform all the operations related with the SSI paradigm. ### European Learning Model (ELM) ELM[^6] is a **multilingual data model** providing a single vocabulary for the description of learning in Europe for Interoperability of Learning Opportunities, Qualifications, Accreditation and Credentials in Europe, developed by the European Commission. “The European Learning Model aims to capture the results of any **non-formal and formal learning across Europe**, as well as the validation of non-formal and informal learning. It is designed to provide a **single** **format to describe** certificates of attendance, examination results, degrees and diplomas, diploma supplements, professional certifications, employer recommendations and any other kind of claims that are related to learning”. Having a single model at European level promotes the free movement of workers and learners through comparability, **portability** and **transparency of data**. “This, in turn, **eases the data exchange** process across Europe as any organization or entity working with learning can make use of the same concepts, making the data understandable even across languages”. Even though the ELM was created for the [Europass Digital Credentials Infrastructure (EDCI)](#user-content-fn-7)[^7] use cases, SSI EduWallets, by defining the verifiable user learning outcomes and the schema that defines this VC, use a portion of [**ELM v3**](#user-content-fn-8)[^8] (which was created based on the W3C standards for verifiable credentials to be interoperable) rather than the entire definition of attributes and properties to provide and describe the key data within the verifiable credential once a user has completed a course or exam. ### Qualification Metadata Schemata (QMS) [Qualification Metadata Schemata (QMS) ](#user-content-fn-9)[^9]refers to a **standard structure** used to describe and represent **detailed information about a specific qualification**. This schema is commonly used in the context of Verifiable Credentials and other qualification management systems to provide additional details about a given qualification or competency. The Qualification Metadata Schemata includes relevant information about the qualification within the verifiable credentials, such as the name of the qualification, the associated education or training level, the learning outcomes or skills acquired, the issuing institution, credits or study hours, among other attributes. The qualification scheme used within verifiable credentials helps to expose the learning results that a user has acquired in a structured and detailed way, so that these verifiable credentials, once presented to third parties, can obtain the results in a structured and detailed way. This is very important when issuing verifiable educational credentials in which learning outcomes, qualifications, learning opportunities, skills and occupations need to be disclosed. Within the QMS it is possible to use[ European Skills, Competences, Qualifications and Occupations (ESCO)](#user-content-fn-10)[^10] to describe what the user achieves once he/she completes a certain assessment. With this data a user can look for a specific job or a higher educational level. The SSI EduWallets try to adapt and make use of this metadata schema in order to define a standard structure for classification of qualifications. ### European Skills, Competences, Qualifications and Occupations (ESCO) **ESCO** is the European multilingual classification of **Skills**, **Competences** and **Occupations**. ESCO works as a **dictionary,** describing, identifying and classifying professional occupations and skills relevant for the EU labor market and education and training. Those concepts and the relationships between them can be understood by electronic systems, which allows different online platforms to use ESCO for services like matching job seekers to jobs on the basis of their skills, suggesting training to people who want to reskill or upskill, etc. The aim of ESCO is to **support job mobility across Europe and therefore a more integrated and efficient labour market**, by offering a “common language” on occupations and skills that can be used by different stakeholders on employment and education and training topics. The SSI EduWallets make use of the ESCO classifications in order to provide the skills and occupations that the users achieve once a course or assessment completion happens. ## System components ### Wallets Wallets are one of the three cornerstones of the SSI paradigm, these are applications that allow users to perform the **exchange of verifiable credentials** with the issuers and verifiers and **store** their verifiable digital credentials and decentralized identifiers that users must use to be compliant with the SSI EduWallets implementation. Those wallets are i.e. developed by third parties following European standards. The implementation of SSI EduWallets provides a web wallet that acts as a demo. It was also tested with another compliant wallet provider “ValidatedID[^11]” for a “proof-of-concept” of a cross-device flow. Both use cases could be tested successfully, so that it was possible to perform the issuance flow. ### Demo web wallet It is a **demo** wallet application that simulates a real wallet workflow and runs based on the[ wallet kit API](#user-content-fn-12)[^12]. It was configured to perform a web wallet flow for performing the issuance and verification. ### User Interface The user interface is the part of the software that is responsible for graphically presenting an abstraction of the logic behind the implementation to users on the learning platforms that implement SSI EduWallets, and where users can interact directly with the implementation. The user interface is implemented based on several views in the frontend that **display the steps of issuing verifiable credentials and verify verifiable presentations**. This graphical framework makes requests to the different APIs to interact and exchange data. ### Wallet Kit API The Wallet Kit API is a third-party integration that is connected with the issuance & verifier APIs and it **is the core of the Web 3.0 & SSI stack** to perform all the operations related to the SSI paradigm and VCs. This API is responsible for DIDs creation, verifiable credential issuance, verifiable presentations, verifiable credential security, the exchange of verifiable credentials, and the demo web wallet. This API is **not directly accessible** but the other APIs are responsible for communicating with it. ### Issuer API (Open API) The issuer API is one of the three cornerstones of the SSI paradigm. This is the main component in charge of handling the user requests to start the verifiable credentials issuance flow and the communication with the wallet kit API (which performs the issuance process). This component must be implemented in each platform to **enable the issuance of verifiable credentials** to users using a compliant wallet. Once the issuer API is integrated in a platform, it can interact directly with the users’ wallets and it interacts with the wallet kit API to issue an educational verifiable credential to the users of the platform, e.g. once they complete a course. ### Verifier API (Open API) The verifier API is one of the three cornerstones of the SSI paradigm. This is the main component in charge of handling the user requests to create a verifiable presentation triggering the start of the verification flow and further on communicating with the wallet kit that performs the verification process. This component must be implemented within any platform to **allow the verification and validation of verifiable credentials** from the users using a compliant wallet. Once the verifier API is integrated in a platform, it can interact directly with the users and connects with the wallet kit API in order to verify if a verifiable credential is valid or not. As soon as the verification process is successful, the platform can use the released data of the verifiable credentials to perform further actions. ## System architecture ### Implementation architecture The implementation of SSI EduWallets is based on the integration of 5 key components. **Wallet KIT API, Issuer API, Verifier API,issuance / verification UI & demo Web Wallet** within a client's infrastructure such as an e-learning platform. These components are in charge of allowing a platform to **issue verifiable credentials & verify verifiable credentials** to/from users who use compatible wallets or within the demo web wallet. Once these software components are integrated, they will communicate and exchange information with the users’ wallets through the [OIDC4VC and SIOP protocols](#user-content-fn-13)[^13]. To integrate these components into an e-learning platform, a microservices-based architecture is provided to run and deploy these components using Kubernetes and Helm Chart. The **issuance and verification UI** components can be integrated as a plugin into the front end of the client, from where platform users can interact with the SSI EduWallets implementation to graphically perform the operations associated with issuing and verifying verifiable credentials. These components make API calls to initiate the operations and establish communication between the platform and the user's wallet. The **demo web wallet** is integrated into the client's platform by cloning the repository, deploying and running the front-end container that runs this web app. From this user interface, any user can use this demo as a custom web wallet that acts as a real web wallet to simulate the workflow of a web wallet with which it is possible to request verifiable credentials, create verifiable presentations, receive, store and manage verifiable credentials. This frontend makes API calls to perform all the operations related to the issuance, verification and storage of the verifiable credentials. The integration of the **issuer and verifier APIs** into the client's platform is done by cloning the repositories and running the Kubernetes pod. These APIs are responsible for handling the user requests they enter through the issuance /verification UI to process and forward them to the wallet kit API. The **wallet kit API** integration into the client's platform is performed by cloning the wallet kit repository, deploying and running the kubernetes pod, which is in charge of performing all the tasks related to the SSI and VCs workflow features like the DID generation, cryptography of the VCs, the exchange of VCs, the communication with EBSI ecosystem, issuance of VCs, reading the VPs, and the verification of VPs. Since the issuance and verification processes can be implemented without any decentralized blockchain system, an **issuer and a user can generate their own DIDs, public and private keys through their wallets**. In case of using a blockchain ecosystem like EBSI, DIDs and key will need to be issued by the blockchain provider. For a “proof-of-concept” the method followed for the creation of DIDs is the "key”-method, which encodes the public key in the DID for the verification. VPs are signed with the private key of both, the issuers of the VCs and the holder or presenter of the VP. Using the public key from the issuers and holder, the verifier is able to fully validate the custody chain for the hole VP. In the SSI EduWallets implementation currently it is using the "key"-method because the EBSI ecosystem is not publicly available yet. ### EBSI architecture The entire stack can be integrated into any educational platform that is to be included in the EBSI ecosystem as a trusted issuer. In order to become an issuer in EBSI, the platform must meet some prior requirements and validation by the [TAO (Trusted Accreditation Organization)](#user-content-fn-14)[^14] and then generate a DID, a private key, and a public key. This . The DID and public key are then stored in the EBSI ledger, a decentralized blockchain database that is synchronized between the nodes that comprise it. Afterwards a trusted authority organization (TAO) will grant permissions to the platform to issue certain types of verifiable credentials, in this case educational verifiable credentials. Once the system is integrated into the EBSI, the Wallet Kit API is responsible for managing all processes within the EBSI ecosystem. Once a user requests the issuance of a verifiable educational credential, a record is stored in the EBSI ledger to ensure traceability. Later, if the user presents this verifiable credential to a third party, the third party can verify the credential's validation and authenticity by simply checking the issuer's DID and public key in the ledger and the user's (VC holder's) DID and public key (resolving the user's DID containing the public key). [^1]: Electronic Identification (2022) Electronic identification, Electronic IDentification - IDentity Verification Solutions. Available at: (Accessed: 02 August 2023). [^2]: Decentralized Identifier parts [^3]: European Commission EBSI verifiable credentials, EBSI Available at: [^4]: Cohen, G. and Steele, O. (2023) Verifiable credentials JSON schema specification, W3C. Available at: (Accessed: 09 August 2023). [^5]: European Commission Verifiable diploma schema, Verifiable Diploma Schema - EBSI Specifications -. Available at: (Accessed: 09 August 2023). [^6]: European Commission (no date b) Introduction to the european learning model (ELM), ELM Browser. Available at: [https://europa.eu/europass/elm-browser/index.html#introduction-to-the-european-learning-model-elm ](https://europa.eu/europass/elm-browser/index.html#introduction-to-the-european-learning-model-elm)(Accessed: 09 August 2023). [^7]: European Commission, Europass Digital Credentials Infrastructure (EDCI), Available at: [^8]: Europass Learning Model Upcoming launch of the European Learning Model V3: Europass, Upcoming launch of the European Learning Model v3 | Europass. Edited by the European Commission. Available at: (Accessed: 08 August 2023). [^9]: European Commission (2020) ‘Publishing of Qualification and Learning Opportunity Data Documentation’. [^10]: ESCO About Esco, ESCO. Edited by the European Commission. Available at: (Accessed: 08 August 2023). [^11]: ValidatedId Validated ID - electronic signature and digital identity providers, Validated ID - Electronic Signature and Digital Identity Providers. Available at: [https://www.validatedid.com/en ](https://www.validatedid.com/en)(Accessed: 08 August 2023). [^12]: Walt.id, Walt.id Wallet Kit, walt.id. Available at:(Accessed: 08 August 2023). [^13]: Kristina Yasuda, Dr. Torsten Lodderstedt ,OpenID Connect for SSI, Available at: [^14]: Issuers trust model EBSI, Edited by the European Commission. Available at: [https://ec.europa.eu/digital-building-blocks/wikis/display/EBSIDOC/Issuers+trust+model+-+Accreditation+of+Issuers ](https://ec.europa.eu/digital-building-blocks/wikis/display/EBSIDOC/Issuers+trust+model+-+Accreditation+of+Issuers)(Accessed: 08 August 2023) # Project workflow ## Educational VC issuance workflow outside EBSI The issuance process will be carried out on the different courses or assessments that a user completes so when a user **completes a course or assessment**, he/she can **claim** a digital certificate or "**verifiable credential**" from the **issuance UI** in which the user can perform the **web wallet flow** in which the user can start the issuance flow with her/his web wallet if the issuer configure it or the **cross device flow** in which the issuance UI generate a **QR code** to start the issuance flow from any user’s compliant wallet through the scan of the QR. The educational VC generated is composed by the data of the user (like the user’s DID), the data of the platform (like the issuer DID), the course data like the ESCO skills and occupations which defines the different skills and occupations that the user has acquired on it. And finally the sign of the issuer. This verifiable credential is generated by the platform (issuer) that implements this system in a standard format defined by the Verifiable Credentials Framework to **be easy to verify and impossible to fake**. The VC (Verifiable Credential) will be digitally generated and **signed by the private key of the issuer** using the Public Key Infrastructure (PKI) so it is cryptographically secure and then it **cannot be falsified** and guarantees that it has been issued **by the defined platform to a defined user** **and also that it has not been altered for no one**. Once the verifiable credential is created, it is sent to the user's wallet through the **OIDC** **protocol**, which the user will accept or not to receive the verifiable credential in her/his wallet. ## Verification workflow outside EBSI Another use case of the implementation is to **verify** any VC received from a user’s wallet, since some operations within the platform may require the presentation of a VC. To carry out this process, the platform (verifier) from the verifier UI **requests the user's wallet** to present a certain type of VC, then the verifier platform communicates with the user’s wallet through the OIDC/SIOP protocol and then the user from her/his wallet chooses which VC or VCs want to present to the verifier platform and once it accepts, a verifiable presentation is created acting as a wrapper of the VC or VCs that will be shared with the verifier platform. Then the verifier platform will carry out a **check process** to check whether that VP is valid or not by checking the sign of the user that is presented in the body of the VP and sign of the VC or VCs that are in each VC with the public keys of both user and issuers encoded in their DIDs. Once the VP is valid the platform can make use of these verifiable credentials. ## SSI EduWallets issuance & verification workflow under EBSI services The incorporation of the SSI EduWallets to a platform using the services of EBSI follows the following process: 1. The SSI EduWallets implementation is **integrated** in the defined platform to allow the **issuance** and the **verification** of verifiable credentials. 2. The platform requests a Trusted Accreditation Organizations (TAO) to be onboarded on the **EBSI** ecosystem as a **trusted issuer and verifier** in order to issue and verify educational verifiable credentials. 3. Once the platform accomplish the requirements it is onboarded on **EBSI** and then it can act as an trusted **issuer** and **verifier**, then the first step as a trusted issuer/verifier is to generate a **DID** (Decentralized Identifier), a **public and private key** and then **store** the **DID** and the **public key** on the **EBSI ledger**. 4. Once the user requests to the platform through the **issuance UI** the issuance of an educational VC for the **completed** course or assessment, the platform **will issue a VC** with the **DID of the user**, the **DID of the platform**, the **sign of the issuer** and the other fields that compose the VC. The issuance could be done by the **web wallet flow** in which the issuer connects with the user’s web wallet if the issuer configure it or through the **scan of the QR code** in the issuance UI to perform the **cross device flow** in which any compliant wallet application can scan the QR code that the issuer UI generates and perform the issuance flow. 5. Through the protocols **OIDC/SIOP** the issuer **exchanges the VC** with the user’s wallet, and then the user decides to accept or reject the VC. 6. Once the user **accepts the VC**, then the user holds it in her/his wallet. 7. If any platform needs a VC from the user in **order to verify** some knowledge, then the user from the verification UI creates a **verifiable presentation** which is a wrapper to share VCs with a verifier. 8. The user from her/his wallet will choose the VCs that the verifier is claiming and then accepts to share the selected VCs with the verifier. 9. Once the verifier receive the VP it verifies that the **VCs are not manipulated** and also check that the public key of the user that is encoded in her/his DID are matched with the **sign of the VP** that was signed by the user private key, also it will check if the signs of the VCs that were signed with the private keys of each issuer **matches the public keys of the issuers that are stored in the EBSI ledger**. *Currently, the EBSI infrastructure is not being used since it is not yet in production but rather in the testing phase and it has not been possible to obtain access to these tests in the different phases of tests that have been carried out. On the other hand, following the EBSI standards and requirements, an implementation has been carried out to be compatible with this system in the future* # Course of the work packages ## Work package 1 - *\* * **Start of the project.** * **Assessment of the project** to begin planning & define the tasks to be carried out. * **Development of a website to inform about the project.** * **Creation of a blog** to summarize the creation process/objectives and the phases of the project. ### Project assessment In this first working package it has been done the **project assessment** where the details and specification of the project were set in order to **plan the tasks** that will be done and the **timeline** during the development of it. Also some implementation **decisions were made,** in this implementation we choose to use of **ESSIF/EBSI infrastructure** and the verifiable credentials standards **instead of the EDCI** infrastructure because EDCI is intended **only for educational purposes** that means that the system is not flexible and interoperable with other type of verifiable credentials. Instead the ESSIF/EBSI make use of decentralized blockchain services which provide a **high availability,security and interoperability**, also within this infrastructure a user with a compliant wallet can handle multiple types of verifiable credentials that can be presented to any third party or verifier. We choose the VCs and EBSI because we believe that the real purpose of the EUDI wallets is to **unify** all credentials in a single place and not force users to use different wallet applications to handle their verifiable credentials, also the verifiable credentials of EDCI that use ELM v3 are now focused on the verifiable credentials data model to be implemented on the ESSIF/EBSI infrastructure to perform the same operations that the europass credentials of EDCI. ### Development of website and blog During this working package it was also developed a **web site** for the purpose of informing everyone about the project and their features. It was also developed a **blog site** where the development process was summarized and in which it explained the objectives and the different phases that involved the development of the project. This work package was developed without any deviations, the major achievements are the decomposition of the project into defined tasks. ## Work package 2 - *\* * **Research about the Self-Sovereign Identity (SSI) paradigm in the context of web technologies.** How it works, Why to use SSI paradigm, advantages over the issuance of verifiable credentials. * **Research about European Digital Identity wallets (EUDI) programme** and their current status & use cases for them. Future of the conformant wallets with the EU standards, review the scope of the project to unify different daily tasks in a single app following defined standards from the EU, focus on the issuance of verifiable diplomas to replace current titles. * **Research about the European Self-Sovereign Identity Framework (ESSIF).** How ESSIF works, how to be conformant with the framework, which advantages ESSIF has, review the interoperability with other systems that follow this framework, privacy concerns. * **Research about European Blockchain Service Infrastructure (EBSI) ecosystem,** How EBSI is implemented, which standards EBSI follow, how blockchain or Distributed Ledger Technologies (DLT) works out, how to be compliant with EBSI ecosystem, current status of EBSI, how EBSI is integrated to be use in a SSI wallet, why to use the EBSI ecosystem. * **Research about the Decentralized Identifiers (DIDs) & Verifiable Credentials (VCs)** Standards that they follow from the W3C, How DIDs are generated, which different methods we have to create a DID, how DIDs are store and where, why the DIDs are essential in SSI, how DIDs can be resolve, which information the DID contains, which types of VCs exist, which standards the VCs follow, how VCs are generated, how VCs are secure & how VCs are signed to be tamper-proof, where VCs are store, how the VCs are shared. * **Research about the current wallets** that are compliant with the ESSIF/EBSI ecosystem to implement a solution that leverages on them. * **Research about the verifiable credential schemas** that exist or are defined by an authority, which types are defined in EBSI ecosystem, which fields are mandatory, how to create own schemas, how to use schemas to verify that the verifiable credentials are compliant with the chosen schema. * **Research about pre-build wallet** **solutions** that follow ESSIF/EBSI ecosystem to make a test pilot and check if its possible to leverage on them. * **Research the implementation of the SSI wallet** that allows verifiable credentials issuance, planning & design the architecture of the implementation. In this work package, the **analysis** of the necessary requirements to carry out the **design** and **implementation** of the system has been carried out. ### SSI In this research has been carried out on the **self-sovereign identity paradigm** in the context of Web 3.0, in this process it has been investigated how this new paradigm works, why to use it and its advantages. In this new paradigm, **the user becomes the owner of their data**, so that there are no third parties that store and manage user data. This paradigm also focuses on a more **decentralized** web, improving the availability of services and making it more **secure** by using blockchain to store information about transactions in a way that cannot be tampered with once they have entered the blockchain and streamline the processes of issuance and verification of verifiable credentials. To make use of this new paradigm, the system is made up of **3 or 4** fundamental parts, the **issuer** of verifiable credentials, which is in charge of issuing a cryptographically secure digital credential that demonstrates certain qualities or knowledge of the user. The **SSI digital wallet** which is the application that **an end user uses** in order to **manage, receive, share and store verifiable credentials**. The **verifier** which is in charge of receiving the verifiable credentials of the user's wallets to **verify** that they have not been manipulated and that they really belong to the person who has shared that credential. And finally within any decentralized blockchain system the **blockchain ledger** in which certain transaction **records** and other data are **stored** so that a verifier can actually verify the authenticity of a verifiable credential through the use of asymmetric cryptography. ### EUDI wallets Research has been carried out on the European Union program for the creation of digital identity (**EUDI**) **wallets** under the European Digital Identity Framework to enable the **cross-border recognition** of government electronic identification (‘eIDs’) to access public services, and to establish a Union market for trust services recognised across borders with the same legal status as the traditional equivalent paper-based processes. In which a user can make these wallets so the verifiable credentials can be stored, such as a unique European identity through the use of electronic IDentification, Authentication and trust Services (eIDAS2), these wallets are developed by third parties under the **supervision and standards of the European Union defined by the** [**Architecture and Reference Framework (ARF)**](#user-content-fn-1)[^1]. To provide people with c**ontrol over their online identity and data** as well as to enable access to public, private and cross-border digital services. The objectives of these are: * To be able to store different types of verifiable credentials which prove something about the user. These verifiable credentials may be stored and shared with third parties to exchange information quickly and securely. * Secure and trusted identification to access online services * Mobility and digital driving license * Health * **Educational credentials and professional qualifications** * Digital Finance * Digital Travel Credential ### ESSIF The **European Self-Sovereign Identity Framework** was developed by the European Union in the context of the SSI paradigm in order to create an implementation framework to make use of EBSI services to solve problems like: * Data acquisition and maintenance * Data processing * Data silos * Lack of data control * Privacy issuer * Lack of universality * Lack of interoperability * Limitations of eIDAS * Lack of certifications The use of ESSIF tries to facilitate cross-border interaction with SSI, make national SSI projects interoperable, integrate existing building blocks such eIDAS, stimulate the transformation of public services, build an identity layer within the new European Blockchain Services Infrastructure (EBSI) and stimulate the development and standardization on global level. This framework is intended to make use of the SSI paradigm in order to avoid that the user’s data is managed by third parties and providing the users the management of all of her/his data and make use of the standards of the **verifiable credentials and decentralized identifiers** within the EBSI services in order to store relevant data on the EBSI ledger and perform operations like the **generation of DIDs, issuance of verifiable credentials or verification of verifiable presentations**. The use of this framework allows us to provide an SSI solution under European standards and rules. ### How ESSIF works In the ESSIF framework it is defined by some actors: the issuer, the holder or user, the verifier and the EBSI ledger. From the issuer’s wallet first it generates a DID and a private and public key par, then the issuer stores the public key and registers the DID on the EBSI ledger, then it can issue a verifiable credential to the users. The holder within its own wallet application generates a private key, a public key and a DID in which the public key is encoded, once the holder make a credential request (CR) to the issuer, the issuer issue a verifiable credential to the user and sign it with its private key. Once the user creates a presentation request (PR) to share a verifiable credential with a verifier, the user creates a verifiable presentation that acts as a wrapper of VCs, this VP is signed with the private key that the user holds in its wallet. Once the verifier receive the VP from the holder then it will check the sign of it with the public key of the user that is encoded on the DID of the user, then check if the hash of the content is equals to verify the authenticity of the VP, once it its verified then the verifier will verify the sign of each VC that the VP contains, to perform this, the verifier check the EBSI ledger in order to get the DID and public key of the issuers that they’re registered on the blockchain ledger, it also will check the validity of each VC in order to verify if the VC was revoked or not and finally it will also check that the hash of the content are the same. Once the VP with the VCs inside passes these verification processes then the VP is valid and authentic and then the verifier can use the data of the VCs. ### EBSI EBSI is an initiative of the European commission to create an ecosystem in order to make use of the SSI paradigm under the european regulations and decentralized ledger technologies, SSI EduWallets is compatible with this infrastructure but it is not used yet, the use of EBSI will enhance all the functionalities related to SSI thats why this project aims to make use of it in a near future. EBSI is builded over the ESSIF, VCDM and DID standards and the ecosystem is made up of several elements: * A Trusted Accreditation Organisations (TAO) which is a trusted entity which is responsible for allowing certain entities to issue verifiable credentials of a certain type. * Trusted verifiable credential issuers (TI) which will be able to issue a certain type of verifiable credential to users, derive from the granting of permissions granted by a TAO. * A trusted issuer registry (TIR) in which trusted issuers are registered to verify that the issuance of verifiable credentials is only made or valid if the issuer is within this registry this register is inside the EBSI ledger. * A Trusted Schema Registry (TSR) where verifiable credential schemas are stored for interoperability by having a common registry for everyone within the ecosystem, this register is inside the EBSI ledger. * Trusted Accreditation Organisation Registry (TAOR) registry that contains information about organizations that can accredit another party (Issuer) to issue certain types of VCs. * A network of nodes which currently serves as a decentralized database using blockchain technologies, in which the DIDs of the issuers and their public keys are stored. * Verifiers who will have access to the ledger to obtain the data of the issuers such as their DIDs,public keys, and access to the registry of trusted issuers and schemas in order to verify if a credential is valid or not. #### [**How DLT works**](#user-content-fn-2)[^2] Blockchain can be used to store and manage a variety of different types of information,such as financial transactions, contracts, or other records. However, unlike a traditional ledger or filing cabinet, a blockchain is distributed across a network of computers, which makes it highly secure and resistant to tampering or modification. A blockchain is a linked list of blocks of information that are cryptographically linked together, forming a chain. the process follow the next steps: 1. Each block contains the hash of the prior block in the chain,keeping the integrity of the set of data in the blockchain. 2. Each block can contain transactions, data and a reference to the previous blocks (creating the chain) 3. Each block has a hash that is derived from the block’s content. 4. Each new block contains the hash of the previous block. Transactions are recorded chronologically and cannot be changed once added to the chain. 5. For blocks to be added to the blockchain, it must be achieved through consensus. When transactions are added to a block, the blocks are validated by the network. Every node maintains an identical copy of the blockchain. The currents status of EBSI is now in pilot testing phase, that why the SSI EduWallets implementation can't be used within EBSI right now, The advantages of using the EBSI ecosystem are the following: * Interoperability between verifiable credentials since their scheme is in a common registry, anyone can know the implementation of a certain type of verifiable credential. * Greater availability and redundancy of data, when using a distributed system. * Greater security by using a blockchain system to store the records of each transaction made, and using the standards defined by ESSIF creating a trust framework. * Use of self-sovereign identity, so that it is the users who own their data and not third parties . * Management of which entities can issue certain types of verifiable credentials. * Revocation of verifiable credentials to a certain user. Through the implementation of SSI EduWallets that make use of the API wallet kit, it is possible to configure and perform onboarding in the EBSI infrastructure. To be compatible with EBSI, it is needed to use the ESSIF framework, and to join the ecosystem, the applicant must meet certain requirements that a TAO defines. ### DID The SSI EduWallets make use of the Decentralized Identifiers in order to provide an unique identifier for anything, A DID is just a long string that does not provide any meaningful information about a natural or legal entity they are a new type of identifier that enables verifiable, decentralized digital identity. The design enables the controller of a DID to prove control over it without requiring permission from any other party. DIDs are URIs that associate a DID subject with a DID document allowing trustable interactions associated with that subject. Each DID document can express cryptographic material, verification methods, or services, which provide a set of mechanisms enabling a DID controller to prove control of the DID. DIDs are used to ensure the authenticity of issuers and holders in machine verifiable documents known as Verifiable Credentials (VCs). DIDs establish a (distributed) public key infrastructure (DPKI) and allow parties to find each other, authenticate and encrypt and verifiably sign data. The DIDs are generated by applications such as the wallets of the users or issuers, through the wallet kit API you can make a call to it to generate a DID in that we can choose with which method and cryptography model to generate it. A variety of “DID methods'', which are different implementations of the DID specification, exist. Considering that DID methods differ in terms of how they are created, registered and resolved, different methods come with different advantages and disadvantages. In this implementation we will be able to generate DIDs through three different methods: 1. **DID:ebsi**: DIDs that are often anchored on Registries, such as EBSI 2. **DID:web:** DIDs that are often anchored to the Domain Name Service 3. **DID**:key: DIDs that do not require Registries because their distribution is based on peer-to-peer interactions. In the SSI EduWallets implementation it makes use of the **DID:key** method because no registries are being used. Normally the DID are generated by the users' wallets so that they are the ones in charge of storing them. In the case of an issuer, the same thing can happen, even though within EBSI infrastructure the issuer can store and register its DID in the EBSI ledger. The information contained in DIDs is given by the DID document information which normally contains the cryptographic form necessary to authenticate the DID controller, metadata, DID controller, context or other attributes. To resolve a DID, it is necessary that this DID be shared with a system that integrates functions for its resolution. Based on the DID generation method, it will be resolved one way or another. With these data, the function in charge of its resolution performs the resolution and the DID document is obtained, which contains all the information of the DID document, such as public cryptographic keys associated with the DID ### VC The SSI EduWallets make use of the Verifiable credentials data model in order to create digital credentials that accredit something to someone. Through the use of the wallet kit API, a data model can be generated manually from a JSON document that will be imported with a name given in it to later allow the creation of verifiable credentials of a certain type defined with the data that is supplied to it. provide by filling in the fields that have been defined in the data model or template earlier. Within the scope of EBSI there are different types of verifiable credentials depending on who they are for. In the case of a normal user who uses EBSI services, in this infrastructure two types of verifiable credentials are defined: verifiable credential to obtain an identity and verifiable educational diploma type credential. To exchange verifiable credentials between the issuer and the user's wallet, the [OIDC4VC protocol](#user-content-fn-3)[^3] is used, which is an authentication and authorization protocol using tokens that allows a secure data exchange between the issuer and receiver, the same occurs in the case of presenting a verifiable presentation, this time using the [OIDC4VP protocol ](#user-content-fn-4)[^4]which is a variation of the previous one to perform the reverse flow from user wallet to platform or verifier. Once verifiable credentials are issued to a user's wallet and accepted by the user, these are stored in the user's wallet in the JSON-LD format or as JWT. Once the credential issuance process is completed before sending it to the recipient user's wallet, the issuer calculates a hash of the credential's content and using its private key encrypts the verifiable credential that is sent to the recipient. Once the verifiable credential is received in the user's wallet, the user can make a verifiable submission to share this verifiable credential with a third party or verifier. Before sending the verifiable presentation to the verifier, the user calculates the hash of the content of the verifiable presentation which contains the verifiable credential to be shared and then using his private key encrypts the verifiable presentation. Once the verifier receives the verifiable presentation, it decodes its public key through the DID of the user and verifies that it can really decrypt the verifiable presentation, then it calculates the hash of the content and checks that it also matches, once the user verification is done, the next thing is to check the verifiable credential, in this case if EBSI is being used the verifier will search the EBSI ledger for the DID and public key of the issuer, if no decentralized system based on blockchain is used then the verifier obtains the public key by decoding the DID of the issuer , once the public key is obtained, the previous process is repeated and if both the presentation and the verifiable credential are valid, then the authenticity of the shared verifiable credential data will be confirmed. ### VC schemas Currently within the EBSI infrastructure there are numerous verifiable credential schemes associated with verifiable credentials of a certain type, even though for real use cases of a user we only find two schemes, the [verifiable credential scheme for identification ](#user-content-fn-5)[^5]and the credential scheme verifiable educational diploma. Logically, outside of the EBSI ecosystem, we do not find any defined scheme that is stored in the same place as it is in the case of EBSI, but this does not mean that there are no defined schemes, there are, the only problem is that they are not all in the same common registry which can cause problems when verifying and trusting these schemes. In SSI EduWallets, when a verifiable credential is created, it has to meet certain requirements. These are that the verifiable credential is defined within a [verifiable attestation credential](#user-content-fn-6)[^6], which defines some mandatory fields which are the following: context, id, type, issuer, issuanceDate, issued, validFrom, credentialSubject, credentialSchema. Once a verifiable credential of a certain type is created, it is subject to the definition of its schema in which certain fields will be defined as mandatory or optional, and once a verifiable credential of a certain type is issued to a user at the moment After presenting it to a verifier, the verifier will carry out a search to obtain the scheme that defines that credential and will go through a validation process in which it will be verified that the definition of the verifiable credential conforms to the defined scheme. This is done through security policies. verification, which the verifier configures. To create schemes for a certain type of verifiable credential, it is only necessary to create a definition of the fields that the verifiable credential will have and define which of these fields are mandatory or not. In this implementation, to define the scheme of the verifiable credential created, the model used in the already existing EBSI schemes has been followed. ### ESSIF/EBSI compliant wallets During the course of this work package it was performed a research in order to find wallets compliant with ESSIF/EBSI infrastructure, we manage to found some wallets like the [web wallet of “walt.id”](#user-content-fn-7)[^7] and “ValidatedID” which are compliant with EBSI and both were tested with the SSI EduWallets implementation allowing the issuance flow, in which these wallets could communicate with the implementation in order to receive and perform the exchange process of verifiable credentials between the issuer and the user’s wallet. The achievements on this work package was the knowledge how this new paradigm of Web 3.0 fits in the current context to the status of the EUDI wallets program, how this implementation works and the architecture behind it. The major problems of this work package was to research about a lot of different approaches and new concepts and unify them to understand how SSI works and the architecture that composes it. This work package was developed without any deviations from the initial planning. ## Work package 3 - *\* * **Setup of the SSI EduWallets architecture** Definition of the architecture that the project will follow, how the project will be implemented, which approach we follow (microservices or other). * **Testing some use cases** for our proof of concept, what will be needed to perform verifiable credentials issuance, how the users are signed up in the platforms through an SSI wallet, how the platform can verify the verifiable credentials that the users send to / share with the platform. * **Issue Verifiable Credentials** How it’ll be issued, which steps are needed to issue a verifiable credential to a user, which fields are essential within verifiable credentials. * **Verify Verifiable Credentials** How the verifiable credentials that the users share with the platform are verified, which steps are needed to verify a VC, what happens after the verification process finishes. * **Setup a basic web interface** to create a test pilot of a real user claiming a verifiable credential to the platform. * **Implementation of the endpoints** for the registration/login using a SSI wallet, issuance request, issuance of a verifiable credential, verifiable credential request, verification of verifiable credentials. * **Setup of the hardware,** which hardware is needed and how it’ll hold the API, how many resources are needed, which budget is needed, how it’ll be configured. In this work package, the construction and start-up of the project architecture on the real infrastructure has been carried out. ### Setup of the SSI EduWallets architecture Since the project is made up of different software components, the most appropriate architecture for this is microservices in which the different APIs will be isolated from each other, but in turn will be interconnected through configuration files. ### Testing some wallets use cases During the course of this work package, proof of concept has been conducted to see if it made sense to implement an implementation based on educational verifiable credentials in the context of SSI. Once a verifiable educational credential design has been decided, tests have been carried out to reproduce the issuance and verification flow that a user would carry out on a platform that uses this system. And it has been verified that both flows have been possible through the Open API and the API wallet kit. ### Issuance process of verifiable credentials The appropriate configuration has been made in the wallet kit API to allow the issuance of verifiable credentials to users of compatible wallets. Verifiable credentials will be issued by the wallet kit API through the issuance API to user wallets. To issue a verifiable educational credential, the user of the platform must complete a course so that he or she can issue the credential with the data of the platform, the user, and the achievements obtained at the end of the course. The required fields of the educational verifiable credential are the following: context, id, type, issuer, issuanceDate, issued, validFrom, credentialSubject, credentialSchema ### Verification process of verifiable credentials The appropriate configuration has been made in the wallet kit API to allow the verification of verifiable submissions from the users' wallets to the verifiers. Once a user interacts with the verification UI components it sends a request to the verification API to connect to the user's waller through the [OIDC4VP](https://openid.net/specs/openid-4-verifiable-presentations-1_0.html) protocol then the users can share the verifiable credentials they have in their wallets by choosing the verifiable credentials from the wallet, then a connection between the user's wallet and the verifier is created thus exchanging a verifiable presentation containing the verifiable credentials the user has chosen to share. Through the verification API, the user's VP is received and it is processed and forwarded to the API wallet kit to be verified, then the API wallet kit performs the verification and returns the result of this together with the credentials to the verification API verifiable. Once the verification process finishes each platform (verifier) decides how to process the information to be used within the platform. ### Basic web interface to perform the issuance process A basic user interface component has been designed and created as a test for the issuance of a verifiable credential. This component consists of a button which makes a request to the API for the issuance of verifiable credentials, which is in charge of starting the issuance flow from this graphic component. ### Implementation of Open API endpoints Adding open API endpoints to the issuance and verification UI components to enable interaction between UI components and user wallets to perform verifiable credential issuance and presentation verification flows verifiable. ### Hardware setup Selection of the server software that will execute the Open API and the API wallet kit, how many resources will be dedicated to it, what budget will be allocated, how it will be configured and what architecture will be followed. For the deployment and start-up of the project, it has been chosen to use a microservices architecture through the use of kubernetes and Helm chart, in the kubernetes configuration file the resources that will be used to start up the entire stack that makes up the project will be established. project, the software on which the different software components will be executed will be on containers through nginx servers The achievements on this work package was to set up an API that allows us to implement an SSI Kit to perform the task of creating DIDs,VC types, issue the VC to the users and verify the VC that the users send to the verifier. The major problems was to read and understand all the documentation about how to implement the workflows with the SSI wallets for issue a signed credential and verify it, also we are trying to get access to the EBSI ecosystem to implement a whole solution based on the Europe Union infrastructure to be fully compliant with European Union standards and ensure interoperability. Our request to the EU/EBSI team is under review. This work package was developed with some deviations to become compliant with the EBSI ecosystem EBSI. ## Work package 4 - *\* * **Implementation of writing the verifiable credentials** to the user‘s wallet, once the verifiable credential is created: how the exchange between platform (issuer) and the user (holder) is performed, how the user stores its verifiable credential in his/her wallet. * **Implementation of the reading a verifiable credential** from a user‘s wallet, How the exchange process between user (holder) and the platform (issuer) is created, how the issuer API verify the verifiable credentials that the user share with the platforms, how the data of the verifiable credentials are manage inside the platforms * **Documentation of the code** that we‘re implementing, how the wallet kit needs to be implemented in our ecosystem (PoC), which API calls are defined in the application, instructions on how to integrate the project in ecosystems. * **Implementation of open API** to serve as an intermediary to communicate the user request with the platform and the issuer wallet. * **Implementation of authentication system** that allows users to signup and sign in on the platforms, integration of an IDP kit to allow users to perform those actions using an SSI wallet. * **Build and link the whole SSI ecosystem** build the open API and integrate the IDP API and SSI wallet API. * **Implementation of the EBSI onboarding** integrating the project with the EBSI ecosystem. * **Implementation of UI components** to perform the actions of sign up / sign in, verifiable credentials issuance, and verification of verifiable credentials. In this work package, the entire implementation of the system has been developed once the different development options derived from the research process previously carried out have been evaluated. ### Store verifiable credentials on the user’s wallet Once a verifiable credential is created through the API wallet kit, it is to be shared from the platform with the user's wallet, some processes need to be carried out to allow its exchange. In this process, the [OIDC4VC](https://openid.net/specs/openid-4-verifiable-credential-issuance-1_0.html#name-introduction) protocol is used for the issuance of verifiable credentials and the [OIDC4VP](https://openid.net/specs/openid-4-verifiable-presentations-1_0.html) protocol for the presentation of verifiable presentations, this protocol is an authentication and authorization protocol based on the [OAuth 2.0 protocol](#user-content-fn-8)[^8], it is used to exchange information between an issuer and a receiver through the use of tokens that are used to verify the identity of the user and thus be able to access the protected resources. Once communication has been successfully initiated between the issuer of the verifiable credential and the user's wallet then the wallet is able to read and interpret the verifiable credential and decide whether or not to store the verifiable credential in its wallet. ### Reading verifiable credentials (Verifiable presentation) Once a user through his wallet wants or needs to share a verifiable credential with a third party or verifier, this from the wallet itself creates a verifiable presentation which acts as a wrapper on the verifiable credentials that he decides to share, to carry out the process To exchange credentials from the user's wallet to the verifier, it is necessary to use the [OIDC4VP](https://openid.net/specs/openid-4-verifiable-presentations-1_0.html) protocol, which is an authentication and authorization protocol based on the OAuth 2.0 protocol, it is used to exchange information between a sender and a receiver through the use of tokens that are used to verify identity and therefore have access to protected resources. Once the verifiable credential is received by the verification platform, it makes a request with the necessary data to the verification API, which will process the received data and forward the request against the API wallet kit, which will be in charge of verifying the verification. verifiable presentation by checking both the user and issuer signatures of the VCs in addition to checking the hashes of the verifiable presentation content. Once the verifiable presentation is validated, the API wallet kit returns the response with the validation result and the content of the verifiable credentials to the verification API and this is in charge of providing the verification UI component on whether the result of the validation is correct or not. Once the content of the validated verifiable credentials has been obtained, each platform can carry out the pertinent processing on that data. ### Implementation of a Open API An Open API was developed in order to serve as an **intermediary** between the Wallet Kit API, which is the core in charge of performing all the tasks related to SSI, such as the creation of DIDs, issuance of verifiable credentials, creation of verifiable presentations and their validation. This API is divided into **two functionalities**, both of which are responsible for communicating directly with user wallets through a graphical interface on the platform that integrates the implementation. The first functionality is **the issuance of verifiable credentials** to the users of the platforms, this is launched through the issuance UI on the platform which initiates the issuance flow. From this UI, a request is made to the issuance API, which processes the request and makes another request to the Wallet kit API. Once it reaches the Wallet Kit API, it processes the request and returns the result to the issuance API, which will later return this same result to the user's wallet already with the verifiable credential. The other functionality is the verification of verifiable credentials, through the verification user interface the user sends a request from the frontend of the platform to the verification API which processes the request and forwards it to the API wallet kit that is It is responsible for making a connection between the user's wallet and the verification API. This makes a redirection to the user's wallet asking for certain types of verifiable credentials, once the user chooses the verifiable credential or credentials that he wants to share with the platform, the user's wallet creates a verifiable presentation with the verifiable credentials or credentials that he has chosen. and it is sent to the verification API which will process the request and forward it to the API wallet kit to be verified. Once verified, the API wallet kit will return the verification result to the verification API, which will manage the presentation data and display the verification result on the platform's verification UI. ### Documentation of the Open API Once the development and implementation of the Open API has been completed, a document has been created that documents the Open API endpoints. This document is in **JSON format** that can be opened by web applications such as "[**swagger.io**](#user-content-fn-9)[^9]" to graphically display the documentation of the endpoints implemented to perform verifiable credential issuance and verification flows. ### Implementation of authentication system Through the implementation of the verification API, it is possible for a user from their wallet to create verifiable presentations and send them against the platform that implements SSI EduWallets, in such a way that a certain type of credential can be configured to be presented in order to to be validated and allow access to the platform. ### Building the SSI EduWallets Once the system components have been correctly developed, the next step is for them to communicate with each other to unite the system and allow the correct use of the implementation. This is done by configuring the configuration files of the wallet kit API to define how it will connect to the open API, which endpoints are available for the open API and the port configuration of the containers where the different APIs such as the wallet kit are executed. API and the Open API. ### Implementation of EBSI onboarding Through the integration of the API wallet kit it is possible and ready to be integrated with the EBSI ecosystem. But because EBSI is still in the testing phase, it is not available in production at the moment, so the system has been implemented outside of any decentralized blockchain service so that the credentials are verified directly with the public keys encoded in the user and issuer DIDs ### Implementation of UI components for issuance and verification To carry out both the issuance and verification flows of verifiable credentials on a platform that implements this system, graphical components are needed, which can be integrated into the platforms, so that each one of them will be in charge of carrying out an operation. The issuance component is in charge of graphically displaying in the platform UI the steps to follow to issue verifiable credentials to users, and the verification component is in charge of graphically displaying in the platform UI the steps to follow to perform the presentation of verifiable credentials. This work package was developed with some deviations while trying to integrate and obtain access to the pilot testing within the EBSI, the major achievement was to perform the issuance and verification through an open API between the user and the issuer. The major problems are related to the steps needed to exchange a verifiable credential with the wallet of the user. ## Work package *5 - \* * **Documentation and final report of the project.** Creation of a general documentation about the project and their features. * **User documentation.** Creation of a user manual to introduce this target audience to the project, this documentation is an abstraction of the technical architecture of the project. * **Developer documentation.** Creation of a developer manual to introduce this target audience to the implementation and integration of the project to convey details to be enabled to reuse the SSI approach. * **Summarize report** creation of a short document to summarize the whole project. * **QMS integration.** How to reuse the Qualification Metadata Schemata in context with wallets. * **ESCO integration**. How to reuse the European Skills, Competency & Occupations Taxonomy in context with wallets. * **Public relation report.** Creation of a public relation report. * **Marketing.** Creation of a marketing report. In this work package, the documentation of the implementation and investigation of the system, marketing and public relations has been carried out. There has also been research on the integration of QMS and ESCO within the educational verifiable credentials. ### Final report of the project In this final report of the project, a general summary of the research, development and implementation process of the SSI EduWallets project is made. ### User documentation A dedicated documentation has been created for users of platforms that integrate SSI EduWallets in order to show how the implementation works so that a user can interact with the system in order to issue and verify verifiable credentials. ### Developer documentation A documentation dedicated to developers has been created in order to document all the technical aspects of the project and put any developer in context of the system architecture in order to allow the development, implementation and deployment of the on any platform. ### Summarize report A brief abstract report on what SSI EduWallets is, how it works, who it is for and a list of project results has been created in order to briefly report what the project is about. ### QMS integration During the implementation of this system, it has been investigated how to integrate the necessary fields in the structure of the verifiable credential in JSON-LD format that makes up the structure of the verifiable credential in ELMv3. Since ELM and its respective components, such as QMS, have been developed solely for the purpose of being used in the EDCI infrastructure on europass, to make use of this system an integration of QMS over the verifiable credentials has been attempted, but in this proof of concept **a full QMS integration has not been done** as this data model has been pre-developed to the verifiable credentials data model and EBSI infrastructure. ### ESCO integration Research has been carried out on **European Skills, Competences, Qualifications and Occupations** and how to integrate this classification system within verifiable educational credentials in order to obtain important metadata about skills, qualifications and occupations that a user achieve through the completion of a learning opportunity in a multilingual format in order to be recognized at a European level and therefore both have a common and interoperable set of learning outcomes. This work package was developed without any deviation. ## Work package *6 - \* * **EBSI assessment.** Onboard to the EBSI ecosystem into the project and validate the data on the European Blockchain Services Infrastructure, compliance to their rules, regulations and standards. * **Check the compliance of SSI & GDPR.** Review the compliance of the SSI paradigm and the GDPR to fulfill the current data protection laws in the scope of the European Union. * **Evaluation of the European Learning Model (ELM)** Matching different data formats to be ELM and wallet kit compliant, compare differences between LOM (IEEE1484.12.1) with[ Learning Opportunities Metadata Schema (LOMS)](#user-content-fn-10)[^10], importance of EQR/QDR, compliance & structure of EU diplomas. * **Evaluation of license models** to distribute the project under certain restrictions & requirements. In this work package a review of IT compliance and data laws protection was done in order to guarantee a legal and correct implementation of the SSI EduWallets project within e-learning platforms. ### EBSI assessment An evaluation of EBSI has been made on which requirements are necessary to be able to be onboarded on the EBSI ecosystem as issuers of verifiable credentials and how to perform the verification of verifiable credentials using the EBSI ledger. It has also been investigated how to proceed as a **trusted issuer** to obtain permission to **issue verifiable educational credentials**, in this case an admission process must be carried out in which the issuer makes a request to a Trusted Accreditation Organisation (TAO) which is in charge of decide whether to accept the new issuer as trusted issuer of a certain type of verifiable credentials. If the TAO accepts the request then the issuer is registered on the Trusted Issuer Registry (TIR) and then the issuer only needs to generate a DID, a private and public key and store the DID and the public key on the EBSI ledger. Through the implementation of SSI EduWallets is possible to be onboarded on the EBSI infrastructure but for now, it is not possible yet due that EBSI is in a test phase and not production ready. ### ELM During the course of the project, an attempt has been made to implement the European Learning Model (ELM) data model to have a data structure under a common framework, in order to obtain detailed skills documentation and complete interoperability between systems that use this system that the European commission has developed, however it has not been possible to perform a complete integration of this data model, since for this proof of concept it does not make sense to create such an extensively detailed verifiable credential with the necessary fields and classes that ELM proposes, so a simpler variant has been developed but making use of some of the classes and fields that this model uses. ### Compliance with GDPR Due that this project is intended to work with the user’s data we had to research about the compliance with the actual data laws in the european scope, we found out that with the new paradigm of SSI the user's data protection is much higher than before and then accomplished the General Data Protection Regulation (GDPR). ### License models The project was evaluated in order to set a license model under certain restrictions and requirements of the project. The criteria considered in order to choose the licenses were encouraging collaboration, reusability, and future development, as well as providing transparency trust, and accountability as users can inspect the code. Keeping the commercial feasibility of the outcome of the project was also an important topic considered. The main developments are licensed under the MIT license. Some other developments are extensions on top of existing projects with already defined licenses like Apache License v2, in this case the original license was kept. General documentation is licensed under Creative Commons. This work package was developed without any deviation. The major achievement was the assessment of the architecture & technical details of the EBSI ecosystem & its impact on GDPR compliance, as well as, identifying relevant ELM data models and their schemata in terms of compatibility. ## Work package *7 - \* * **Reports submission** of the project when it's finally finished. * **Submission of all reports** * **Bill/invoices submission** In this last work package is performed the submission of the documentation reports and the bill/invoices submission. [^1]: European Commission (no date c) The European Digital Identity Wallet Architecture and Reference Framework, Shaping Europe’s digital future. Available at:[ https://digital-strategy.ec.europa.eu/en/library/european-digital-identity-wallet-architecture-and-reference-framework](https://digital-strategy.ec.europa.eu/en/library/european-digital-identity-wallet-architecture-and-reference-framework) (Accessed: 08 August 2023). [^2]: European Commission(10/06/2021) , EBSI Architecture, explained. Available at: %3Dv2\&usg=AOvVaw2MiBXC8nid8v-sO-TU2Jyn\&opi=89978449 [^3]: T. Lodderstedt,K. Yasuda, T. Looker (03/02/2023), OpenID for Verifiable Credential Issuance. Available at: [^4]: O. Terbu, T. Lodderstedt,K. Yasuda, T. Looker (21/04/2023), OpenID for Verifiable Presentations - draft 18. Available at: [^5]: European Commission (no date d) Verifiable attestation for ID, Verifiable Attestation for ID - EBSI Specifications -. Available at: (Accessed: 12 August 2023). [^6]: European Commission (no date c) Verifiable attestation, Verifiable Attestation - EBSI Specifications -. Available at: (Accessed: 09 August 2023). [^7]: Walt.id ,Walt.id Web wallet, walt.id. Available at: (Accessed: 08 August 2023). [^8]: Oauth OAuth 2.0, OAuth. Available at: (Accessed: 12 August 2023). [^9]: Swagger API development for everyone, Swagger. Available at: (Accessed: 12 August 2023). [^10]: European Commission (2020) ‘Publishing of Qualification and Learning Opportunity Data Documentation’. # List of final project results | | | | | ---------------------------------------------------- | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | | *Interim Report* | *CC BY-SA 4.0* | *Interim report of the project in which the main functionalities and implementation are described.* | | *Final Report* | *CC BY-SA 4.0* | *Final report of the project in which all the components, architecture and functionalities of the project are described.* | | *Developer Documentation* | *CC BY-SA 4.0* | *Technical documentation for developers.* | | *User Manual* | *CC BY-SA 4.0* | *Non-technical documentation for end users.* | | *One Pager* | *CC BY-SA 4.0* | *Project summary.* | | *External Communication* | *see Final Report* | *Stakeholder communications.* | | *SW project result part\_1: Demo Application* | *MIT* | [*https://gitlab.com/ssi-edu-wallets/demo-application*](https://gitlab.com/ssi-edu-wallets/demo-application) | | *SW project result part\_2: Wallet Proof of Concept* | *Apache License 2.0* | [*https://gitlab.com/ssi-edu-wallets/wallet-proof-of-concept*](https://gitlab.com/ssi-edu-wallets/wallet-proof-of-concept) | | *Verifiable Credentials, QMS & ELM* | *CC BY-SA 4.0* | *Documentation about Verifiable Credentials in context of the European Learning Model (ELM) and the Qualification Metadata Schemata (QMS).* | | *ESCO API* | *CC BY-SA 4.0* | *Documentation about API integration of ESCO taxonomy provided by the European Commission.* | # Application of the project results in practice The project results are intended to be used in the context of Learning Management Systems (LMS) or educational online platforms or environments (i.e. Learning Ecosystems). Documentations as well as public repositories provide a general introduction on how to set up a wallet architecture based on the concepts of Self-Sovereign Identity (SSI). Our research also contains schemata layouts to be compliant to * the European Learning Model (Europass / ELM) via Verifiable Credentials as well as * the European Self Sovereign Identity Framework (ESSIF) and European Blockchain Service Infrastructure (EBSI) # Public Relations/ Networking We introduced parts of our research to the public at the LearnTec Congress in June 2023 in Karlsruhe for the first time (speaker slot). Our future goal is to spread the technology approach throughout Europe and to establish close links with government agencies - a process we have already begun. # Own project website Website: # Planned activities after the end of the project * We plan to reach out to commercial partners & customers which are interested in integrating the EduWallet approach with us. * Together with existing clients (e.g. Universities) we want to establish “best-practice” use cases to promote the EduWallet approach to the public. * We will work on our main landing page () and make more details & main code repositories available. * We plan to advertise the landing page () via SEM (Google Ads). # Suggestions for further developments by third parties EBSI integration: EBSI was not integrated due to lack of responsiveness from the EBSI group team providing any answer to our application (beta test, wave 3). The system should be compatible with EBSI and integration should not be complex. *Prerequisite: EBSI is publicly available* Signup authentication system: An OIDC workflow to exchange credentials between the wallets and 3rd party platforms was implemented. However, direct signup and sign-in of new users into 3rd party platforms using information from the wallet still needs to be developed. This implementation should be straightforward in a very similar way to the use cases already implemented within the project. *Prerequisite: It could be done as soon as there are trustable issuers, such as identity providers from different EU countries, that generate VerifiableIDs containing basic user data.* SSI EduWallets provides third parties with a kit to issue and verify verifiable credentials from the users’ wallets. With this implementation, any third party can leverage the use of VCs and DIDs to perform: 1. Issuance and verification of new types of verifiable credentials. 2. Cross-device flow on verification. 3. Improve or create new schemas to add more information to the verifiable credentials issued. 4. SSI onboarding creating user profiles into a platform streamlining the process & the user data privacy. ## Real use case example Another possibility would be to implement a real use case scenario ready to go live into production. This real use case scenario could consist, for example, of generating a certificate after completion of an online course. This certificate would be generated by the LMS and could be verified by the educational institution. The implementation of this use case would require the following steps: 1. Define what needs to be stored in the credential (event/course name, teacher/presenters, small description, unique identifier for the online content, etc.). The definition of this data is the schema of the certificate and will depend on the final intended use of the credentials. Some validation would be needed to make sure the course details are not tampered in the LMS platform once exported into the credential and the contents in the credential cannot be confused with any other course. 2. Validate the personal data of the user (email, name, etc.). Depending on the real use case of the certificate. Some validation of the identity of the user may be required since the wallet will only communicate the DID from the user, maybe it is needed to validate that the owner of the DID is the same user registered in the learning platform and not someone else. This process can be challenging since no official institution is issuing identity credentials at the moment, but this process may not be totally necessary if the level of trust needed for the credentials is not high. 3. Implementation of a fully featured wallet APP (this may be avoided if the decision is made to work with a wallet provider owning an already existing wallet solution) 4. The restrictions for the registration with EBSI could be a blocker, for this reason, the usage of a DID system without registries, based on peer-to-peer interactions (key), could be more suitable. This would require different actors: * Educational technology provider implementing an LMS or similar platform. * Educational institution making use of the real use case with real users. * Wallet provider, implementing a ready-to-production wallet APP. # Documental sources Cohen, G. and Steele, O. (2023) *Verifiable credentials JSON schema specification*, *W3C*. Available at: (Accessed: 09 August 2023) Electronic Identification (2022) *Electronic identification*, *Electronic IDentification - IDentity Verification Solutions*. Available at: [https://www.electronicid.eu/en/blog/post/eidas-2-0-what-can-companies-expect-from-it/en ](https://www.electronicid.eu/en/blog/post/eidas-2-0-what-can-companies-expect-from-it/en)(Accessed: 02 August 2023). ESCO *About Esco*, *ESCO*. Edited by the European Commission. Available at: (Accessed: 08 August 2023). Europass Learning Model *Upcoming launch of the European Learning Model V3: Europass*, *Upcoming launch of the European Learning Model v3 | Europass*. Edited by the European Commission. Available at:[ https://europa.eu/europass/tr/news/upcoming-launch-european-learning-model-v3 ](https://europa.eu/europass/tr/news/upcoming-launch-european-learning-model-v3)(Accessed: 08 August 2023). European Commission(10/06/2021) , *EBSI Architecture, explained.* Available at: European Commission (2020) ‘Publishing of Qualification and Learning Opportunity Data Documentation’. European commission (2022) *Official Legal Text*, *General Data Protection Regulation (GDPR)*. Available at: (Accessed: 12 August 2023). European Commission, *Europass Digital Credentials Infrastructure (EDCI),* Available at: . European Commission *EBSI verifiable credentials,* EBSI Available at: European Commission *European Blockchain Services Infrastructure*, *Home - EBSI -*. Available at: (Accessed: 09 August 2023). European Commission *Introduction to the european learning model* , *European Learning Model for Stakeholders | Europass*. Available at: (Accessed: 09 August 2023). European Commission *Verifiable diploma schema*, *Verifiable Diploma Schema - EBSI Specifications -*. Available at: (Accessed: 09 August 2023). European Commission (no date b) *Introduction to the european learning model (ELM)*, *ELM Browser*. Available at:[ https://europa.eu/europass/elm-browser/index.html#introduction-to-the-european-learning-model-elm](https://europa.eu/europass/elm-browser/index.html#introduction-to-the-european-learning-model-elm) (Accessed: 09 August 2023). European Commission (no date c) *The European Digital Identity Wallet Architecture and Reference Framework*, *Shaping Europe’s digital future*. Available at: (Accessed: 08 August 2023). European Commission (no date c) *Verifiable attestation*, *Verifiable Attestation - EBSI Specifications -*. Available at: (Accessed: 09 August 2023). European Commission (no date d) *Verifiable attestation for ID*, *Verifiable Attestation for ID - EBSI Specifications -*. Available at:[ https://ec.europa.eu/digital-building-blocks/wikis/display/EBSIDOC/Verifiable+Attestation+for+ID](https://ec.europa.eu/digital-building-blocks/wikis/display/EBSIDOC/Verifiable+Attestation+for+ID) (Accessed: 12 August 2023). European Commission (no date d) *What is EBSI*, *What is EBSI - EBSI -*. Available at: [https://ec.europa.eu/digital-building-blocks/wikis/display/EBSI/What+is+EBSI](/ssi-edu-wallets) (Accessed: 10 August 2023). European Learning Model v3 *european-commission-empl/European-Learning-Model* Edited by European Commission. Available at:[ https://github.com/european-commission-empl/European-Learning-Model](https://github.com/european-commission-empl/European-Learning-Model) Kristina Yasuda, Dr. Torsten Lodderstedt ,*OpenID Connect for SSI,* Available at: . O. Terbu, T. Lodderstedt,K. Yasuda, T. Looker (21/04/2023), *OpenID for Verifiable Presentations - draft 18.* Available at: . Oauth *OAuth 2.0*, *OAuth*. Available at: [https://oauth.net/2/ ](https://oauth.net/2/)(Accessed: 12 August 2023). Pastor Matut, C. and Du Seuil, D. *Understanding the European self-sovereign identity framework (ESSIF)*, *PPT*. Available at: [https://www.slideshare.net/SSIMeetup/understanding-the-european-selfsovereign-identity-framework-essif ](https://www.slideshare.net/SSIMeetup/understanding-the-european-selfsovereign-identity-framework-essif)(Accessed: 08 August 2023). Sporny , M., Longley , D. and Chadwick , D. *Verifiable credentials data model V1.1*, *W3C*. Available at: (Accessed: 08 August 2023). Sporny, M. *et al.* *Decentralized identifiers (DIDs) v1.0*, *W3C*. Available at: (Accessed: 08 August 2023). Swagger *API development for everyone*, *Swagger*. Available at: [https://swagger.io/ ](https://swagger.io/)(Accessed: 12 August 2023). T. Lodderstedt,K. Yasuda, T. Looker (03/02/2023), *OpenID for Verifiable Credential Issuance.* Available at: . ValidatedId *Validated ID - electronic signature and digital identity providers*, *Validated ID - Electronic Signature and Digital Identity Providers*. Available at: [https://www.validatedid.com/en ](https://www.validatedid.com/en)(Accessed: 08 August 2023). Walt.id ,*Walt.id Wallet Kit*, *walt.id*. Available at:[ https://github.com/walt-id/waltid-walletkit ](https://github.com/walt-id/waltid-walletkit)(Accessed: 08 August 2023). Walt.id ,*Walt.id Web wallet*, *walt.id*. Available at:[ https://github.com/walt-id/waltid-web-wallet ](https://github.com/walt-id/waltid-web-wallet)(Accessed: 08 August 2023). 1. European Commission\_.\_ *Different data paradigms along the web generations*, *EBSI*. Available at (Page 3): ↑ 2. European Commission *The European Digital Identity Wallet Architecture and Reference Framework*, *Shaping Europe’s digital future*. Available at: (Accessed: 08 August 2023). ↑ 3. Sporny, M. *et al.* *Decentralized identifiers (DIDs) v1.0*, *W3C*. Available at: (Accessed: 08 August 2023). ↑ 4. Sporny , M., Longley , D. and Chadwick , D. *Verifiable credentials data model V1.1*, *W3C*. Available at: (Accessed: 08 August 2023). ↑ 5. Pastor Matut, C. and Du Seuil, D. *Understanding the European self-sovereign identity framework (ESSIF)*, *PPT*. Available at: [https://www.slideshare.net/SSIMeetup/understanding-the-european-selfsovereign-identity-framework-essif ](https://www.slideshare.net/SSIMeetup/understanding-the-european-selfsovereign-identity-framework-essif)(Accessed: 08 August 2023). ↑ 6. European Commission (2020) ‘Publishing of Qualification and Learning Opportunity Data Documentation’. ↑ 7. ESCO *About Esco*, *ESCO*. Edited by European Commission. Available at: (Accessed: 08 August 2023). ↑ 8. European Commission *European Blockchain Services Infrastructure*, *Home - EBSI -*. Available at: (Accessed: 09 August 2023). ↑ 9. European commission (2022) *Official Legal Text*, *General Data Protection Regulation (GDPR)*. Available at: [https://gdpr-info.eu/ ](https://gdpr-info.eu/)(Accessed: 12 August 2023). ↑ 10. Electronic Identification (2022) *Electronic identification*, *Electronic IDentification - IDentity Verification Solutions*. Available at:[ https://www.electronicid.eu/en/blog/post/eidas-2-0-what-can-companies-expect-from-it/en](https://www.electronicid.eu/en/blog/post/eidas-2-0-what-can-companies-expect-from-it/en) (Accessed: 02 August 2023). ↑ 11. Sporny, M. *et al.* *Decentralized identifiers (DIDs) v1.0*, *W3C*. Available at: (Accessed: 08 August 2023). ↑ 12. Decentralized Identifier parts ↑ 13. European Commission *EBSI verifiable credentials,* EBSI Available at: ↑ 14. Cohen, G. and Steele, O. (2023) *Verifiable credentials JSON schema specification*, *W3C*. Available at:[ https://www.w3.org/TR/vc-json-schema/](https://www.w3.org/TR/vc-json-schema/) (Accessed: 09 August 2023) ↑ 15. European Commission *Verifiable diploma schema*, *Verifiable Diploma Schema - EBSI Specifications -*. Available at:[ https://ec.europa.eu/digital-building-blocks/wikis/display/EBSIDOC/Verifiable+Diploma+Schema](https://ec.europa.eu/digital-building-blocks/wikis/display/EBSIDOC/Verifiable+Diploma+Schema) (Accessed: 09 August 2023). ↑ 16. European Commission (no date d) *What is EBSI*, *What is EBSI - EBSI -*. Available at: (Accessed: 10 August 2023). ↑ 17. European Commission (no date d) *What is EBSI*, *What is EBSI - EBSI -*. Available at: (Accessed: 10 August 2023). ↑ 18. European Commission (no date b) *Introduction to the european learning model (ELM)*, *ELM Browser*. Available at:[ https://europa.eu/europass/elm-browser/index.html#introduction-to-the-european-learning-model-elm](https://europa.eu/europass/elm-browser/index.html#introduction-to-the-european-learning-model-elm) (Accessed: 09 August 2023). ↑ 19. European Learning Model v3 *european-commission-empl/European-Learning-Model* Edited by European Commission. Available at:[ https://github.com/european-commission-empl/European-Learning-Model](https://github.com/european-commission-empl/European-Learning-Model) ↑ 20. European Commission *Introduction to the european learning model* , *European Learning Model for Stakeholders | Europass*. Available at:[ https://europa.eu/europass/en/node/2128 ](https://europa.eu/europass/en/node/2128)(Accessed: 09 August 2023). ↑ 21. European Commission, *Europass Digital Credentials Infrastructure (EDCI),* Available at: . ↑ 22. Europass Learning Model *Upcoming launch of the European Learning Model V3: Europass*, *Upcoming launch of the European Learning Model v3 | Europass*. Edited by the European Commission. Available at: (Accessed: 08 August 2023). ↑ 23. European Commission (2020) ‘Publishing of Qualification and Learning Opportunity Data Documentation’. ↑ 24. ESCO *About Esco*, *ESCO*. Edited by the European Commission. Available at: (Accessed: 08 August 2023). ↑ 25. ValidatedId *Validated ID - electronic signature and digital identity providers*, *Validated ID - Electronic Signature and Digital Identity Providers*. Available at: (Accessed: 08 August 2023). ↑ 26. Walt.id, *Walt.id Wallet Kit*, *walt.id*. Available at:[ https://github.com/walt-id/waltid-walletkit](https://github.com/walt-id/waltid-walletkit) (Accessed: 08 August 2023). ↑ 27. Kristina Yasuda, Dr. Torsten Lodderstedt ,*OpenID Connect for SSI,* Available at: ↑ 28. Issuers trust model EBSI, Edited by the European Commission. Available at: (Accessed: 08 August 2023) ↑ 29. European Commission (no date c) *The European Digital Identity Wallet Architecture and Reference Framework*, *Shaping Europe’s digital future*. Available at: (Accessed: 08 August 2023). ↑ 30. European Commission(10/06/2021) , *EBSI Architecture, explained.* Available at: ↑ 31. T. Lodderstedt,K. Yasuda, T. Looker (03/02/2023), *OpenID for Verifiable Credential Issuance.* Available at: ↑ 32. O. Terbu, T. Lodderstedt,K. Yasuda, T. Looker (21/04/2023), *OpenID for Verifiable Presentations - draft 18.* Available at:[ https://openid.net/specs/openid-4-verifiable-presentations-1\_0.html](https://openid.net/specs/openid-4-verifiable-presentations-1_0.html) ↑ 33. European Commission (no date d) *Verifiable attestation for ID*, *Verifiable Attestation for ID - EBSI Specifications -*. Available at:[ https://ec.europa.eu/digital-building-blocks/wikis/display/EBSIDOC/Verifiable+Attestation+for+ID](https://ec.europa.eu/digital-building-blocks/wikis/display/EBSIDOC/Verifiable+Attestation+for+ID) (Accessed: 12 August 2023). ↑ 34. European Commission (no date c) *Verifiable attestation*, *Verifiable Attestation - EBSI Specifications -*. Available at: (Accessed: 09 August 2023). ↑ 35. Walt.id ,*Walt.id Web wallet*, *walt.id*. Available at: (Accessed: 08 August 2023). ↑ 36. Oauth *OAuth 2.0*, *OAuth*. Available at:[ https://oauth.net/2/](https://oauth.net/2/) (Accessed: 12 August 2023). ↑ 37. Swagger *API development for everyone*, *Swagger*. Available at:[ https://swagger.io/](https://swagger.io/) (Accessed: 12 August 2023). ↑ 38. European Commission (2020) ‘Publishing of Qualification and Learning Opportunity Data Documentation’. ↑ # Experiments report This [project](https://gitlab.com/eduplex-api/text-analysis) explored various **AI experiments** using different natural language processing (**NLP**) models to inform the development of Retrieval-Augmented Generation (**RAG**) systems for the AI assistant, the AI tutor, and the recommendation engine. The experiments provided valuable insights that guided the selection of the most effective approaches for these applications. This document explores methodologies, technologies, and AI-driven solutions for exploiting course content in a digital environment. We show how the ESCO standard (European Skills, Competences, Qualifications, and Occupations) can be used to map courses to skills, and by integrating advanced semantic matching and embedding models, we explore state-of-the-art technologies for enhancing educational resources and learner outcomes. The structure of this document is designed to guide readers from foundational concepts to advanced AI applications. We begin by introducing the key concepts and data sources used, including insights into the ESCO ontology. We follow by exploring mechanisms of semantic matching and the innovative approaches employed to recommend and integrate ESCO skills into course classification. The last section is dedicated to the design and implementation of different proofs-of-concept, detailing the architecture, data processing pipeline, and the critical choices made in embedding model selection. These choices inform a range of AI applications, from course content search and annotation to generative question-answering and hybrid retrieval systems. Finally, conclusions and recommendations are presented, summarising the most important findings and focusing on practical deployment. # Key concepts This section examines the key necessary techniques to understand the work carried out, aiming to provide a general overview of the methods used rather than the technical details. We start with the BM25 algorithm, widely accepted as one of the best methods to search text literals. We follow by giving a general explanation of embeddings, LLMs and two methods used to help retrieve (bi-encoder) and classify (cross-encoder) text. Finally, we introduce the concept of Retrieval-Augmented Generation. 1. **BM25**: Best Matching 25 \[1] is a commonly used ranking algorithm in information retrieval that evaluates the relevance of a document (e.g. paragraph, text snippets or chunks) to a query by considering how frequently the query terms appear in the document (term frequency, TF), how rare those terms are across all documents (inverse document frequency, IDF), and normalising for document length to avoid bias toward longer documents, making it effective for keyword-based search in systems like search engines and text databases. This algorithm is the baseline for academic and industrial applications, and it is implemented in search engines such as Elasticsearch1, Apache Lucene2 and Solr3. 2. **Embeddings**: These are built on the transformer architecture and represent text as dense numerical vectors in a high-dimensional space. These embeddings capture the meaning and context of words, phrases, or sentences by processing text through layers of self-attention and feed-forward neural network mechanisms. Unlike static embeddings such as Word2Vec, transformer embeddings are contextual, meaning the representation of a word adapts based on the surrounding words. For instance, in "a light bulb" versus "travel light," the word "light" is represented differently because the model understands the context in which it occurs. This ability to model relationships bidirectionally, considering both preceding and following words, allows transformer embeddings to encode rich and nuanced language information.\ Embeddings are fundamental to many NLP applications due to their capacity to generalise across tasks. Pre-trained transformer models like GPT, T5, or DistilBERT learn embeddings from vast quantities of text data, capturing linguistic patterns, syntax, and semantics. These embeddings can then be fine-tuned or directly applied to tasks such as text classification, machine translation, or summarisation. The transformer architecture's attention mechanism ensures that relationships between all words in a sentence are considered, enabling the embeddings to represent complex dependencies. 3. **Bi-encoder**: A bi-encoder architecture takes two separate neural networks to independently encode two inputs (e.g., a query and a document) into fixed-size vector representations, which are then compared (e.g., via cosine similarity) to measure their semantic relevance or similarity. An alternative to this configuration is to use the same neural network to encode both inputs (e.g., query and document), which generally is referred to as a shared-weight bi-encoder, this configuration is the one used in this project to which we refer by its general bi-encoder name. 4. **Cross-encoder**: A cross-encoder architecture takes two inputs (e.g., a query and a document) that are processed together by a single neural network, allowing it to directly evaluate their interaction and output a relevance score or classification, often achieving higher accuracy than bi-encoders at the cost of slower performance. 5. **LLM**: Large Language Models (LLMs) are advanced neural networks designed to process, understand, and generate human-like text, images, videos and other digital signals. They are built with billions of parameters and trained on massive datasets, using techniques like pretraining (on general text) and fine-tuning (for specific tasks). LLMs leverage the transformer \[2] architecture for efficient language understanding and generation. Some key applications include: chatbots, virtual assistants, and content creation; language translation and summarisation; code generation and debugging; Domain-specific tasks (e.g., legal, medical, scientific analysis); education and personalised learning tools. The challenges of LLMs include bias in training data, ethical concerns about misuse, high computational costs, and their limited ability to understand context and meaning. 6. **RAG**: Retrieval-Augmented Generation is a cutting-edge approach in machine learning that integrates the know-how of information retrieval systems, which fetch relevant documents or facts from external knowledge bases, with generative language models, which process and synthesise this information alongside the user’s query to generate coherent and context-aware responses, making it ideal for applications like question answering, summarisation, and conversational AI. 1 2 3 # Data sources During this work, we explored diverse data sources, both open and closed. For example, we made extensive use of advertised job data to better understand skills and their relationship with course offerings. We used standards like ESCO, as shown in Figure 1, to create our database for skills descriptors.

Figure 1, ESCO data preparation

Regarding course data, we explore several open collections available on the Internet to understand the data ecosystem. However, for our experiments and demonstrators, we used two primary course data sources. WBS provided the first, and it consisted of courses with three main data fields, namely course title, course description and course learning goals. Hochschule Schmalkalden provided the second-course data set. It consisted of more complex content because it included additional information such as credit number, course semester, instructor, etc. An example is shown in Figure 2.

Figure 2, Hochschule Schmalkalden MS course data

# First demonstrator: ESCO ontologies and semantic matching ESCO \[3] describes in detail the skills required for an occupation. It currently provides around 13900 skills linked to 3008 descriptions of occupations. It aims to support job mobility across Europe by improving labour market integration and efficiency. ESCO provides a common vocabulary and ontology developed around three themes: occupations, skills, and qualifications. In fact, it is based on three types of terms that support language analysis for text matching, namely: * **preferred terms:** They are not used for any other occupation or skill and thus are unique. Preferred terms are the ones that best represent an occupation or skill. * **non-preferred terms:** They can be synonyms but also spelling variants, declensions, abbreviations, etc. They are regularly used by jobseekers, employers, or education institutions to refer to concepts related to preferred terms. * **hidden terms:** Terms commonly used to address occupations but are considered outdated, misspelt, or politically incorrect. They are useful for indexing, searching, and text mining purposes but are invisible to the end users. ESCO ontology, definitions, and structures aim to improve the matching of the profiles of learners, job opportunities, and learning resources. ESCO pillars are depicted in Figure 4. ![Figure 4, ESCO three pillars and profiles integration](https://3619117356-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FPnUjsAMcXqV3uh7MX1l3%2Fuploads%2FiVTbHtZQpwlUCM6tVqbL%2F3.png?alt=media) ## **ESCO Skill and competences pillar** To better understand how the ESCO skills and competences work, we will focus on the *skills and competences* pillar. At the very top level, four main concepts are found, namely: knowledge (K); Language skills and knowledge (L); Skills (S); and Transversals skills and competences (T). Each one of these top levels contains a hierarchy of concepts that is better illustrated by taking one of them to exemplify its usage. For instance, taking the ***“Agile project management”*** skill and walking up the hierarchy, we have the following path: *knowledge > business, administration and law > business and administration > management and administration > ICT project management methodologies > Agile project management.* Furthermore, every skill has the following properties: * Description: textual skill description * Alternative labels: alternative concept that represent a similar skill(s) * Skill type: top hierarchical level * Skill reusability level * Relationships * Broader concepts: related wider concepts * Optional for: related optional concepts * Concept Uri: ontological URI with a link to the hierarchical concept and all its related information. Skill’s textual descriptions are used to embed a sentence-transformers model and then maps sentences and paragraphs to a 384-dimensional dense vector space that can be used for tasks like clustering or semantic search. The process follows the steps described below. * ESCO Skills descriptions are “embedded” using SBERT\[4] with a model called “all-MiniLM-L6-v2”\[5]. * Course module contents are used as queries against the embedded model (trained with skills descriptions) * then we retrieve the ESCO skill most closely associated with the query (module contents). * The resulting model and index are saved to disk and used for similarity search based on the cosine similarity (other similarity metrics such as k-nearest-neighbor from Faiss can be easily used). Notice that using all the textual descriptions to create an embedding of ESCO skills and competences allows us to match course textual module contents and obtain a similarity metric from this comparison. Figure 5 shows a typical course description from a university offering. This is a structured document that contains the module name, professor name, qualifications targets, module contents, teaching methods, and additional relevant information for students. ![Figure 5, Applied Computer Science M.Sc. course specification from the Schmalkalden University of Applied Sciences](https://3619117356-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FPnUjsAMcXqV3uh7MX1l3%2Fuploads%2FoVeRHjtXtxGdilxJbPIM%2F4.png?alt=media) From this document, we extract the relevant fields for our prototype, as shown in Figure 6. ![Figure 6, courses module name and module contents](https://3619117356-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FPnUjsAMcXqV3uh7MX1l3%2Fuploads%2FwToLVs9OVmbNAT78hRa5%2F5.png?alt=media) Notice that the third column in the above figure is the module contents, this is the data field that we use to obtain a best match against ESCO skills descriptions. Thus, we use the course description from the “Agile Software Development” module to obtain the best ESCO skill match. Table 1 shows a typical matching output, including the top 5 results.

Table 1, top 5 results from best match semantic similarity

## ESCO skills recommendation and embedding selection One of the main issues we encountered was the large number of embeddings available to vectorise the course’s content, the ecosystem currently provides more than seven thousand\[6] models to choose from, focusing on sentence similarity topics. To determine which model to use, we used a subset of 50 WBS courses and carried more than a hundred simulations to determine the best model and which part of the course text to use. The first question we wanted to answer was to determine which section of a course was the best predictor for finding skills. To achieve this, we used WBS course text split into three categories: course title, course description and course goals. Then we combined the text into different components, for instance, we merge the course title text and the description text, title and goals, goals and descriptions, and so on. This is depicted in Figure 7. We used all these text snippets to match against ESCO skills description and calculated its semantic similarity based on the cosine metric using a fixed typical embedder (all-MiniLM-L6-v2). Then, the top ten semantic similarity responses were selected and then averaged into groups of the top 3, 5 and 10. ![Figure 7, WBS course section analysis](https://3619117356-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FPnUjsAMcXqV3uh7MX1l3%2Fuploads%2FNZYn7EmPySaqKbQ8CGwi%2F6.png?alt=media) From this experiment, we learned that the best course descriptor to predict skills was the course learning goals followed by the course description, noticed that the combination of learning goals and course description produced poor results, and we hypnotise that this is due to the introduction of new text that increases text noise and dispersion. ### Em**bedding model selection** The second question we wanted to answer was to determine which embedding models should be used. As previously mentioned, there are more than seven thousand models to choose from. Moreover, these models are changing and improving all the time, so we needed to establish a methodology that would allow us to compare embedders’ performance against each other to determine which embedding is the best to use to suggest ESCO skills. Figure 8 shows a subset of the model’s comparisons carried out. This subset was selected by taking the top performers in standard benchmarks such as the HuggingFace leaderboard\[7] for semantic textual similarity (STS). As a criterion for model selection, we suggest focusing on the average score of the top 3 results together with the overlapping score. In this case, the overlapping score represents model consistency or robustness, for instance, the gte-large model shows that 30% of the suggested skills were the same using the text for course description and learning goals. All in all, ***gte-large*** is the best performer and the ***all-MiniLM-L6-v2*** is the worst (see Figure 8). These results should be interpreted correctly in the sense that gte-large uses a large context window, so it is slower, thus, more computationally expensive than all-MiniLM-L6-v2. Thus, there is a trade-off between model performance and speed. ![Figure 8, WBS embedding model selection](https://3619117356-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FPnUjsAMcXqV3uh7MX1l3%2Fuploads%2FYPC6fwvVS4OCb97sV7Us%2F7.png?alt=media) The third question we wanted to answer is what skills should be suggested to users based on ESCO. As we have seen previously, the same course can produce different skill suggestions based on the text snipped that it is used (i.e. title vs description vs learning goals). Our overall recommendation is to use the intersection set of suggested skills, as depicted in Figure 9. ![Figure 9, WBS ESCO skill final suggestion](https://3619117356-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FPnUjsAMcXqV3uh7MX1l3%2Fuploads%2FHUEl8D37Lu6ObG3MPwdJ%2F8.png?alt=media) # Software design As a proof of concept, an **automated system** was implemented in the prototype to match learning opportunity descriptions and titles with the **ESCO taxonomy**. This system enhances the categorization process for content providers by leveraging AI tools, streamlining the experience and improving accuracy. The project demonstrated the practical benefits of integrating advanced NLP techniques into educational content management workflows. ## Software license The source code for the site is licensed under the [**MIT license**](https://gitlab.com/eduplex-api), which you can find in the [LICENSE](https://gitlab.com/eduplex-api/text-analysis/-/blob/main/LICENSE/README.md) file. ## Architecture overview ![System architecture overview](https://3619117356-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FPnUjsAMcXqV3uh7MX1l3%2Fuploads%2FaP3dPIe8wDg9pjzYNfZ9%2F9.png?alt=media) ## Base data processing pipeline ![Text processing pipeline](https://3619117356-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FPnUjsAMcXqV3uh7MX1l3%2Fuploads%2FYWaNU6XMpY6EZtpUvhEM%2F10.png?alt=media) # Endpoints Sbert\_eduplex ## Endpoints usage ### Find ESCO skill best match based on different text (label and description) and languages (English and Deutsch) GET (deprecated) GET (deprecated) GET (deprecated) ### Vectorise a text string based on the default LLM GET (not publicly available) POST (publicly available, Authorization header is needed) Request example: ```json { "vectorise": [ "Text A to vectorise", "Text B to vectorise", "Text C to vectorise" ] } ``` @return: a JSON file with dimension, llm model, text to vectorise, and the numerical vector for eact text within the vectorise array ### Compute skills similariry of existing skills in english GET POST (Authorization header is needed) Takes a JSON file with a pre-defined language and computes similarity scores for skills based on ESCO descriptions Similarity score is computed for each pair of skills. Currently only english is supported Request example: ```json { "language": "en", "skill": "Manage musical staff", "skills_eval": [ "Manage musical staff", "supervise correctional procedures", "apply anti-oppressive practices" ] } ``` Response example: JSON file with cosine similarity scores ```json { "0": { "base_skill": "Manage musical staff", "base_skill_id": 1, "eval_skill": "Manage musical staff", "eval_skill_id": -1, "score": -1 }, "1": { "base_skill": "Manage musical staff", "base_skill_id": 1, "eval_skill": "supervise correctional procedures", "eval_skill_id": 2, "score": 0.840195 }, "2": { "base_skill": "Manage musical staff", "base_skill_id": 1, "eval_skill": "apply anti-oppressive practices", "eval_skill_id": 3, "score": 0.788353 } } ``` ### Vectorise a text string based on the default LLM GET POST (Authorization header is needed) Takes a JSON file with a pre-defined serialised LLM and retrieves similarity scores for skills based on ESCO descriptions. Scores are retrieved from a previously vectorised model containing all vectors for all ESCO's skills trained with ESCO descriptions. Thus vector embeddings are not computed but retrieved, but semantic similarity is computed. ```json { "language": "en", "skill": "Manage musical staff", "skills_eval": [ "manage musical staff", "supervise correctional procedures", "apply anti-oppressive practices" ] } ``` @return: JSON file with cosine similarity scores ### Vectorise a text string based on the default LLM GET (not publicly available) POST (Authorization header is needed, also including boolean param `sanitize_all`) Takes a JSON. ```json { "language": "en", "title": "Manage musical staff", "description": "Manage musical staff", "learning_goals": "Manage musical staff" } ``` @return: JSON file with top k matches for title, description, and learning goals # Setup Sbert\_eduplex ## Setup Sbert\_eduplex ### Directory structure: * The file called `server.py` is the main file to run the application * `.env` should be used to write down the initial parameters of the application. Logs should be automatically created. * `requirements.txt` contains the requirements ### How to develop sbert with docker * Run `docker-compose-socket-sbert.yml` (change the image version to force a rebuild) * This will make the service available in [localhost:5000](http://localhost:5000/sbert_en/query=Examine%20images%20taken%20by%20telescopes%20in%20order%20to%20study%20phenomena%20and%20objects%20outside%20Earth%20atmosphere) without SSL #### How to build sbert service for production * Change the image version in `docker-compose-sbert.yml` (optionally use dockerfile `docker/child/Dockerfile` for a quick build without updating pip dependencies or `docker/Dockerfile` for a slower build from empty image) * Run **docker-compose up** will create a new version tagged image * Login to AWS ECR `aws ecr-public get-login-password --region us-east-1 --profile prdedupl | docker login --username AWS --password-stdin public.ecr.aws/eduplex_api` (change `prdedupl` with the name of your aws cli or `default` if you do not have many profiles) * Push the image `docker push ` (image\_tag from yml file) ## technical notes * in pycharm there may be problems with dockers permissions, after locally installing Dockers and pycharm plugin, the following may be needed: ``` sudo chmod a+rwx /var/run/docker.sock sudo chmod a+rwx /var/run/docker.pid ``` # AI Applications This section presents several demonstrators that were created to demonstrate our findings. For instance, Figure 12 shows the final API to suggest ESCO skills for course annotation using embedders, a bi-encoder and cross-encoder. This API follows a client/server architecture and has been deployed without major issues for over a year. ![Figure 12, ESCO skill API recommendation](https://3619117356-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FPnUjsAMcXqV3uh7MX1l3%2Fuploads%2FiFh07VOhjMow8N4dglVy%2F11.png?alt=media) ## Generative Q\&A response (bi-encoder and generative responses) A bi-encoder with a combined generative response integrates a dual-encoder retrieval system with a generative model to efficiently retrieve relevant information and generate natural, context-aware responses. The process begins when a user submits a query. A bi-encoder handles retrieval by encoding the query and documents into embedding vectors using transformer-based models like BERT or RoBERTa \[4]. The query embedding is compared to the precomputed document embeddings using a similarity metric, such as cosine similarity or dot product, to retrieve the most relevant documents. After retrieval, the system processes the documents to extract the most relevant information. This step may involve selecting specific sections (chunking), summarising lengthy content, or filtering out irrelevant details to ensure concise and focused input for the generative model. Next, the retrieved content and the original query are passed to a generative model to produce a coherent response. The query and retrieved-context are combined into a structured input. Finally, the system post-processes the generated response to clean up redundant or unnecessary text, improve clarity, and ensure factual accuracy if additional verification is applied. This approach combines the efficiency of bi-encoders for retrieval with the natural language generation capabilities of generative models. The result is a system that delivers accurate, dynamic, and contextually relevant responses while leveraging the scalability of precomputed document embeddings and ensuring the generative output remains grounded in retrieved information. ![Figure 13, AI generative response example](https://3619117356-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FPnUjsAMcXqV3uh7MX1l3%2Fuploads%2F5CW0gkAO2lOsKrPt1lzq%2F12.png?alt=media) ## Retrieval Augmented Generation (RAG) for Documents Q\&A Retrieval-Augmented Generation (RAG) is a framework designed for tasks like document Question-and-Answering (Q\&A). It combines a retrieval component to fetch relevant information and a generation component to produce coherent responses. The process begins when a user provides a question or query. The system uses a retrieval model to fetch relevant documents or passages from a knowledge base. This knowledge base is pre-processed into a searchable format, often using embeddings. Retrieval typically relies on methods like Dense Passage Retrieval (DPR) or vector similarity searches, such as FAISS \[5], but traditional approaches like BM25 can also be used. To retrieve the most relevant documents, the input query is first converted into an embedding using a model like BERT. The query embedding is then compared to the document embeddings using measures like cosine similarity to rank relevance. The top-k most relevant documents are selected, where 𝑘 is a parameter that can be tuned. Sometimes, the retrieved documents are further processed or condensed to ensure that only the most relevant portions are used. This step may include techniques like chunking, summarisation, or filtering. The refined information is then passed to the generation module. The generation model takes the query and the retrieved documents as input to produce the final answer. The input to the generator typically combines the query and the retrieved content, such as in the format: "Question: \. Context: \." The generator, often a model like GPT or T5, uses this input to craft a coherent and contextually relevant response. The output may also undergo post-processing to clean or format the response for the user. Optionally, RAG systems can include a feedback loop to refine their performance. This might involve re-ranking the relevance of retrieved documents or fine-tuning the generative model based on user feedback. RAG offers several advantages. It can scale to work with large, dynamic knowledge bases, which can be updated independently of the generative model. Grounding the generation in specific retrieved content, it reduces the likelihood of generating hallucinated or inaccurate answers, ensuring that responses are both accurate and dynamic. This approach is widely applied in document Q\&A systems, customer support, open-domain Q\&A, and research assistance, making it an effective framework for integrating retrieval and generation in a single pipeline. ![Figure 14, RAG for documents Q\&A](https://3619117356-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FPnUjsAMcXqV3uh7MX1l3%2Fuploads%2FPZdUfcifBgOM8BHYmlZ7%2F13.png?alt=media) ## Hybrid retrieval system: RAG using BM25 and cross-encoder for Q\&A BM25, cross-encoders and RAG can be combined to create a highly effective and efficient Q\&A system by leveraging their complementary strengths. Together, they can provide accurate retrieval, precise ranking, and coherent response generation. The process starts with BM25, which acts as the initial retrieval mechanism. When a user submits a query, BM25 retrieves a broad set of potentially relevant documents from the database base. This step is computationally efficient and quickly narrows down the vast pool of documents to a manageable subset based on keyword matching and term frequency scoring. Next, the retrieved documents are re-ranked using a cross-encoder for more precise scoring. The query and each document are paired and jointly encoded by a transformer model, which analyses their relationship in detail. The cross-encoder assigns a relevance score to each query-document pair, ranking the documents based on how well they answer the query. This step ensures that the most relevant and contextually appropriate documents are prioritised. Finally, the top-ranked documents are passed into a RAG framework for response generation. The user’s query, along with the retrieved and re-ranked documents, is fed into a generative model like GPT, Llama, T5, etc. The model uses the provided context to produce a coherent, natural language response that is both accurate and grounded in the retrieved documents. By combining these three methods, the system benefits from BM25’s speed for initial retrieval, the cross-encoder’s precision for re-ranking, and RAG’s ability to generate detailed and contextually appropriate answers. This integration ensures that the Q\&A system is both efficient and capable of delivering high-quality responses. Notice that this proposed setting is a variation of the previous architecture, where instead of using BM25, a bi-encoder was used. Thus, the question is which architecture should be used. In simple terms, currently, there is no right answer to this question. The dataset and preprocessing (e.g. chunking) will largely influence the results. ## Course examiner, marking student’s work For our last demonstrator, we used llama3.2 LLM model running on-premises under Ollama \[6]. In practice, Ollama works like ChatGPT, but it is open source, and it can be run within a closed system, making it ideal when privacy is a priority and flexibility using other models is required. Additionally, Ollama offers several state-of-the-art models with a thriving community that ensures that the model is updated often. For this demonstrator, we used the llama3.2 as a base model to create our own llama3.2, which has been instructed to act as an examiner to grade students’ exam questions. For visualisation and testing purposes, a UI was created which is shown in Figure 15. Initial tests are encouraging, we have tested this system with Q\&A from tests provided by Hochschule Schmalkalden. We have observed that the shorter the student’s responses, the more accurate the assigned grade is. We are confident that these preliminary results can be improved. Thus, we plan to continue this work, focusing on training the models with well-curated data and the creation of an API that can be integrated into other services. ![Figure 15, EduPlex Examiner](https://3619117356-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FPnUjsAMcXqV3uh7MX1l3%2Fuploads%2FitlsrnTdSUQFziSXEbdd%2F14.png?alt=media) # Conclusions Information retrieval technology has advanced greatly, and current systems continue to improve rapidly. However, there are still technical barriers and constraints to implement fully functional and deployable systems. Training custom neural network (e.g. LLMs) from scratch give the best possible results, but they require large data, specialised hardware, technical human expertise, and they are economically costly. Thus, the most common use of this technology is based on the subscription model with companies such as OpenAI (e.g. ChatGPT). Given the constraint mentioned above, one common alternative is to use LLMs that have been pre-trained and optimised to carry out tasks within a specific context. Then, these models can be used to train our models as a downstream task. There are open issues such as privacy, copyrights and IP, bias, and other risks that need to be assessed when implementing AI systems, particularly when new legal frameworks such as the EU AI Act (Regulation (EU) 2024/1689) will affect the deployment of AI systems. # Recommendation The following are our main recommendations: * Understanding the problem. Given the many available models and their derived versions, it is essential to select the right model for the tasks at hand. Natural language processing models are generally grouped based on the problem they solve, such as text classification, questions and answers, translation, text generation, clustering, sentence similarity, etc. Framing the problem in the right setting will help to select the best model. * Understanding benchmarks and their use: models are assessed using benchmark datasets for specific tasks. For instance, the Sentence Text Similarity (STS) benchmark is commonly used for semantic text similarity evaluations. These benchmarks can use metrics such as the Person’s correlation coefficient to quantify how well a model’s score aligns with human judgments. Benchmarks can specify other factors such as model size, memory usage, embedding dimensions, maximum token capacity, etc. All these factors are relevant to selecting the best model under certain constraints. * Understanding the technology. A good grasp of embeddings, LLMs, encoders, and information retrieval in general would help build custom pipelines that maximise system performance. * Understanding model performance based on language. Most models have a performance bias towards English. If a model works well in English, this may not be true in another language, such as German. * Understanding how and when to apply translation. Several models perform very well in the translation task; this may allow the translation of all text to English, perform the necessary AI task (matching and retrieval, question and answering, etc.) in English, and then translate the results back to the target language. This will often perform better than working on a non-English language directly. * Understanding text chunking. Models allow to choose a fix length text size for chunking. The chosen size will have a big impact on the model’s performance and its capacity to retrieve relevant passages. Splitting text into meaningful chunks using a semantic parser based on paragraphs or semantic units rather than a fixed size will ensure that text retains relevant context. * Understanding model parameters. Models have several parameters that will affect the outcomes. For example, the LLM temperature parameter will increase the randomness of a model response in the question-and-answer task. If replication of results is important, this parameter should be set to zero. Otherwise, the response will include variations which are difficult to replicate. * Understand your data and how to balance it. Selecting a representative data sample to carry out tests and experiments is important. If data selection is unbalance models can perform well in tests, but this behaviour would not necessarily be translated to the whole data. This is even more relevant if training a model is necessary. * Data preprocessing. The cleaner the data, the better the models will perform, several models have archived comparable results. Thus, effort may be better used in curating data than model parameter tuning. * Know how to use zero-shot and few-shot learning on RAG systems. Use the few-shot learning by providing clear task instructions in the prompt without any specific examples. In the few-shot instance, a few examples of the desired task should be directly included in the prompt. * Finally, we recommend that the deployment planning of a system such as the one discussed in this document should incorporate an adequate risk assessment to mitigate any bias that may be introduced. # Bibliography | \[1] | S. Robertson and H. Zaragoza, “The Probabilistic Relevance Framework: BM25 and Beyond,” *Foundations and Trends in Information Retrieval,* vol. 3, no. 4, pp. 334-385, 2009. | | ---- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | \[2] | A. Vaswani, S. Noam, N. Parmar, J. Uskoreit, L. Jones, A. N. Gomez, L. Kaiser and I. Polosukhim, “Attention Is All You Need,” arxiv.org, 2017. \[Online]. Available: . | | \[3] | ESCO, “Multilingual classification of European Skills, Competences, and Occupations,” \[Online]. Available: . | | \[4] | Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer and V. Stoyanov, “RoBERTa: A Robustly Optimized BERT Pretraining Approach,” 2019. | | \[5] | M. Doueze, A. Guzhva, C. Deng, J. Johnson, G. Szlvasy, P.-E. Mazaré, M. Lomeli, L. Hosseini and H. Jégou, “The Fais library,” Meta, 2024. \[Online]. Available: . | | \[6] | Ollama, “Ollama,” \[Online]. Available: . | | \[7] | J. Devlin, W\.-W. Chang, K. Lee and K. Toutanava, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” arXiv, 2018. | | \[8] | J. Devlin, W. Chang, K. Lee and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” *CoRR,* 2018. | # Recommendation Engine The **Recommendation Engine** project was designed to help users **discover learning opportunities** tailored to their individual needs and preferences, enabling self-paced learning and enhancing user engagement. The system leverages insights from [Text Analysis](broken://pages/eGsdR92t7rE9Pbw4f6Gu) experiments and the ESCO taxonomy to accurately match learning opportunities with user profiles. Recommendations are presented through the [Playlist Builder](https://docs.eduplex.eu/nuxt-listing/) and different visualization dashboards, providing an intuitive and personalized way to explore relevant content. The prototype successfully demonstrated the integration of all components, showcasing the ability to generate recommendations based on user profiles. This project highlights the potential of combining advanced text analysis with structured taxonomies to deliver impactful and user-centric learning experiences. # Reporting and predictive analytics The **Reporting and Predictive Analytics** project was designed to display key user **metrics** captured by the **User Behavior Tracker** with clear and understandable visualizations. These visualizations aim to assist content providers and platform administrators in understanding user interactions and engagement, enabling data-driven decisions to improve learning experiences. The project **successfully developed basic visualizations**, demonstrating the value and utility of the tracked metrics. By providing an intuitive way to interpret user data, the project highlights the potential of leveraging analytics for optimizing content delivery and user engagement. This work establishes a foundation for further enhancements in reporting and predictive capabilities. # LRS User Journey Visualizer The LRS **User Journey Visualizer** project aimed to develop a tool for displaying user journeys in a simple and effective manner based on interactions with learning content. By visualizing user flows, the tool intended to provide content providers and platform administrators with valuable **insights into user behavior** and learning patterns. The project explored multiple visualization approaches derived from xAPI statements to represent these journeys. Although various visualization techniques were tested in the prototype phase, none were deemed sufficiently effective to achieve the project's objectives. Additionally, plans to extend LRS functionalities were deprioritized due to time constraints and their relatively low impact on the core goals. Despite these challenges, the project provided valuable lessons and highlighted the complexities involved in representing user journeys, offering a foundation for future work in this area. # AI Tutor - RAG system ## RAG System Overview Our Retrieval-Augmented Generation (RAG) system is a robust framework designed to enhance the accessibility and usability of learning content. The system operates through three interconnected processes.

RAG Overview created by EduPLEx

First, during the **ingestion phase**, the learning content is processed into structured text chunks, which are then stored in an OpenSearch index to enable efficient retrieval. Next, in the **retrieval phase**, the system leverages the capabilities of OpenSearch to identify and retrieve the most relevant text chunks based on user queries, ensuring accuracy and relevance. Finally, the **answer generation phase** utilizes a Large Language Model (LLM) to synthesize coherent and contextually appropriate responses, drawing from the retrieved content. This integrated approach ensures that users receive precise and informative answers tailored to their needs, making the system a powerful tool for knowledge exploration and learning. # LLM-augmented Retrieval and Ranking for Course Recommendations ## Goal To evaluate the effectiveness of a recommendation pipeline leveraging LLM capabilities. To test how well an LLM can generate synthetic course title recommendations based on user profiles (job title, skills, and goals), and to assess the quality of course retrieval through vector search and LLM-based relevancy re-ranking. ## Data WBS modules data, synthetic data. ## Method/Approach Multi-stage pipeline combining LLM-generated synthetic course recommendations, vector-based retrieval using OpenSearch, and LLM-based relevancy scoring for re-ranking. Evaluation on 100 synthetic user profiles, analyzing cosine similarity scores and LLM-based relevancy. ## **Results** Overall, OpenAI 3-large showed the best retrieval quality, with higher overall and top-1 relevance scores, followed by OpenAI 3-small. SBERT showed the weakest performance in this setup. ## Evaluation Metrics Cosine similarity scores, relevancy scores (heatmap, averages). ## Conclusions OpenAI text-embedding models provided the best retrieval quality. SBERT showed weaker performance, with a bit lower average relevance scores at rank 1 (0.49 compared to 0.52 for OpenAI models).\ The experiment highlighted a key limitation: we cannot distinguish whether low scores are due to a weak retrieval system or a lack of relevant candidates in the database. ## More details about the recommendation system The recommendation system operates by first creating a user profile that includes a set of **user skills**. Due to the absence of actual user data, these skills are **generated synthetically**. Based on this synthetic skill user profile, a large language model (LLM) is employed to produce potential course titles. These **generated titles** may include hypothetical or non-existent courses, serving as proxies for the types of learning opportunities that align with the user's skills. Once these synthetic course titles are generated, they are utilized as input queries to **semantic search** for real course titles that closely resemble the generated ones. This search identifies **existing courses** that are similar in content or focus to the synthetic titles. After retrieving these matches, a **re-ranking** process is applied to refine their relevance and prioritize the most suitable options. The system ultimately provides personalized **course recommendations** by leveraging synthetic data, advanced natural language generation, and retrieval techniques.

Created by the EduPLEx team

See also: * EDX-576 ## Optimize OpenSearch search to obtain more relevant results when searching via title From 2000 synthetic course titles we have performed 100 queries with opensearch. We see a much better performance in openai than in sbert.
The score is slightly better using a search just in the title field only, than when searching in a combination of title + learning goal (see below more details):
See also: * EDX-569 # Retrieval of course candidates when searching via title. \ \ **Goal**\ To optimize the retrieval of relevant course candidates based on similar course titles.\ \ **Data**\ WBS data for 2136 modules, including title, product category, education goal, and learning goal.\ \ **Method/Approach**\ A random sample of 100 titles from the dataset was used as queries. Vector search was done on two different fields: (1) Title and (2) Combined title and learning goal. Three embedding models were tested: SBERT, OpenAI text-embedding-3-small, OpenAI text-embedding-3-large. Vector search replicated locally with FAISS using the same OpenSearch parameters (M=24, ef\_search=100, ef\_construction=128).\ \ **Results**\ The OpenAI 3-large model had the best retrieval scores for both the title-only search (0.92 at k=6) and when combining the title and learning goal (0.85 at k=6). The OpenAI 3-small model wasn’t far behind. SBERT struggled in both cases, showing the lowest scores (0.57 and 0.48 at k=6).\ \ **Evaluation Metrics**\ Retrieval quality: Cumulative Gain (CG at k=1 to k=6)\ \ **Conclusions**\ Results highlight the better retrieval effectiveness of OpenAI text embedding model 3-large, particularly at higher ranks, when searching on title and learning goal combined. In contrast, including the learning goal slightly decreased SBERT’s performance, possibly due to noise or irrelevant information that the embeddings struggled to handle effectively. # Answer Generation Evaluation **Goal**\ To evaluate the generator component of the RAG system, assessing how well the responses answer the question and are supported by the provided context.\ \ **Data**\ Set of questions, retrieved chunks, and generated responses from Kritisches Denken (serviceID 2699) and Agiles Mindset (serviceID 2700). 20 content-related questions for each course.\ \ **Method/Approach**\ Replicating the RAG pipeline, doing a vector search in OpenSearch to retrieve relevant chunks (context). The retrieved chunks and query were passed to the LLM, which generated an answer. The generated answers were evaluated by the LLM for relevance and groundedness.\ \ **Results**\ For both KD and AM, the average relevance score was 0.89, indicating that most answers were highly relevant to the queries. Groundedness scores varied: KD (0.75); Agiles Mindset (0.83).\ \ **Evaluation Metrics**\ Answer Relevance: Evaluates whether the final response addresses the entirety of the user’s question.\ Groundedness: Evaluates if the generated response is based on the context provided by assessing the information overlap between each sentence of the response and relevant parts of the context.\ Scores produced by the LLM are mapped to a range from 0 to 1 (0 = not relevant/not grounded, 1 = relevant/grounded) and averaged to get an overall score.\ \ **Conclusions**\ Most answers were highly relevant to the queries. Outliers with low scores revealed two main issues: retrieval failures where the correct chunk wasn’t retrieved, and content limitations where the available context was insufficient to fully answer the question. # Chunk Size and Retrieval Evaluation **Goal**\ Determine the chunk size that yields the best retrieval performance for our RAG system\ \ **Data**\ Modules: Kritisches Denken , Agiles Mindset (2699, 2700)\ Chunk sizes: 100 and 150\ 20 content-related questions for each course.\ \ **Method/Approach**\ Sentence splitting for chunking\ SBERT for question encoding\ OpenSearch with vector search and retrieval (hnsw params: M=24, ef\_search=100)\ \ **Results**\ Chunk size 100 best for Kritisches Denken. Chunk size 150 best for Agiles Mindset. 150 chosen for standardization\ \ **Evaluation Metrics**\ Retrieval quality: Cumulative Gain (CG at k=1 to k=6)\ \ **Conclusions**\ Standardizing to a chunk size of 150 led to consistent improvement in retrieval quality. However, retrieval performance should be monitored as new courses are added to ensure the chosen parameters remain optimal. # Chunking Techniques – Splitters ## **Goal** Choosing chunking strategy ## **Data** Module 'Kritisches Denken'. Specific long chunk that was identified as problematic in a previous evaluation because the chunk was not retrieved for various questions. ## **Method/Approach** Splitters tested:\ • Sentence Splitter\ • Semantic Splitter\ • Human best-try Splitter\ • Semantic Double Merging Splitter\ • Simple Dot Splitter\ Tested for chunk sizes 100,150, and 200. Sbert vector embeddings for similarity search. ## **Results** With the sentence splitter, the best retrieval quality reached 65% (CG at k=5) with a chunk size of 100. The semantic splitter, with a certain combination, reached 70%, in most cases around 50-60%. Semantic double merging achieved 65% at the best, and the simple dot splitter 62%. ## **Evaluation Metrics** Retrieval quality: Cumulative Gain (CG at k=1 to k=6) ## **Conclusions** The sentence splitter was selected for its balance between retrieval quality and simplicity. The semantic splitter reached the highest retrieval quality at some point, but the right combination of parameters buffer and breakpoint need to be found, adding complexity to the system without providing a big gain on performance. *** ## Optimizing chunking techniques To ensure that relevant information is retrieved and passed to the LLM for generating contextually appropriate responses, large documents are broken down into smaller pieces of text (chunks). **Sentence Splitter** A straightforward technique that avoids cutting sentences prematurely. The splitter tries to keep sentences and paragraphs together. Therefore, compared to the most simple character-based splitter, there are less likely to be hanging sentences or parts of sentences at the end of a chunk. **Semantic Splitter** Organizes text based on semantic similarity. Instead of chunking text with a fixed chunk size, the semantic splitter uses embedding similarity to ensure that a text chunk contains sentences that are semantically related to each other. It is useful for identifying coherent and related chunks of information within a larger body of text. However, it needs an embedding model, a measure of similarity, and it relies on setting the appropriate parameters. **Semantic Double Merging** It extends traditional semantic chunking by adding a second pass that merges chunks to create more content-rich units. Initially, text is divided based on semantic coherence using measures like percentiles or standard deviations. In the second pass, the algorithm evaluates the similarity between the current chunk and a chunk two positions ahead. If a strong cosine similarity is found, all three chunks—the current one and the next two—are merged, even if the immediate chunks are not textually similar. Since this approach helps retain broader context, it is best suited for scenarios where understanding the overall meaning is more important. ### **Our Findings** When we tested these three techniques on our question and answering case data we observed only a slight improvement from the sentence splitter to the semantic splitters. Considering that the implementation of semantic splitters would increase complexity and resources (additional embedding calculations) in our system without adding sufficient benefit, we opted for using a sentence splitter. Moreover, optimizing chunk size with the sentence splitter yields sometimes results as good as with the semantic splitter, at least for our current use case. *** ## Experimental Setup **In Short:** The experiments showed that chunk size positively impacts the retrieval quality of our AI Tutor's RAG system. We tested chunk sizes of 100 and 150 tokens and found that each had its own advantages depending on the course content. Ultimately, we chose to standardize on a chunk size of 150, which leads to a noticeable improvement in retrieval quality across our test cases. **Goal**: To determine the chunk size that yields the best retrieval performance, optimizing for the highest cumulative gain metric (CG at k). **Chunking Technique:** We used a Sentence Splitter for chunking (derived from previous experiments), which segments documents based on sentence boundaries to maintain context, controlled by the specified chunk size (in tokens) and chunk overlap. An overlap parameter of 5 was applied, meaning each chunk included a 5-token overlap with the previous chunk to capture context continuity. The chunks were indexed in our OpenSearch Index with two variations: 100 and 150 chunk sizes. **Vector Search and Retrieval:** Questions were encoded using the SBERT model (thenlper/gte-large). The index was built in OpenSearch using the following hnsw parameters: M=24, ef\_search=100, ef\_construction=128. Vector searches were performed with a size parameter set to retrieve a maximum of either 6 or 10 results per query. Relevant chunks were labeled 1 if they contained the answer to the question, 0 otherwise. **Data** Two prototype courses were used: * **Kritisches Denken** * ServiceID: 2701 (Chunk size = 100), Total chunks: 102 * ServiceID: 2699 (Chunk size = 150), Total chunks: 88 * **Agiles Mindset** * ServiceID: 2702 (Chunk size = 100), Total chunks: 201 * ServiceID: 2700 (Chunk size = 150), Total chunks: 158 Each course was evaluated using 20 content-related questions to measure retrieval quality. ### Results and Conclusion * For *Kritisches Denken* (KD), the optimal chunk size was 100, with improved results observed at the top 4 results (CG at k=4). * For *Agiles Mindset* (AM), a chunk size of 150 provided better results overall. The retrieval is able to find the relevant chunk in 70% of the cases within the top 3 positions. Despite course-specific variations, a chunk size of 150 was selected for consistency. This new chunking strategy, combined with optimized retrieval parameters, improves notably the retrieval quality compared to previously indexed chunks of different sizes (ServiceID 2677 and 2675) where the best CG achieved was 50% for AM and 40% for KD. We should however continue monitoring retrieval performance as more courses are added to ensure the chosen chunk size and retrieval parameters remain optimal. **Retrieval quality for different chunk sizes**.

Cumulative Gain: measure of total relevance of the retrieved list. Here it indicates the percentage of cases where the relevant chunk was found in the k position.

**OpenSearch scores of retrieved chunks by rank position.**

Scores of Retrieved Chuks in the offering "Kritisches Denken"

Red data points indicate relevant chunks. Chunk size is 150.

Scores of Retrieved Chuks in "Agiles Mindset" offering

**Test Questions**

Test questions.

*** ## Some extra details First experiments show using the xapi chunk strategy delivers better results than the simple pdf page chunk strategy. Maybe we can think of parsing the chunks during the index process (to remove useless chunks if exists) \ A second experiment with 2 different courses, is showing that xapi performs better than pdf. We noticed that sometimes the chunks do not contain all the needed context to answer all questions and we are getting some questions that are not answered. 1. Future skills\ ![](https://3619117356-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FPnUjsAMcXqV3uh7MX1l3%2Fuploads%2Fgit-blob-fe599049e5d0691c9cad663dbb2fa6c16c65837f%2Fimage.png?alt=media) 2. Agiles mindset\ ![](https://3619117356-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FPnUjsAMcXqV3uh7MX1l3%2Fuploads%2Fgit-blob-7f9e99c119c2fe487507227b529a72cdf8e3fa44%2Fimage.png?alt=media) 3. Kritisches denken\ ![](https://3619117356-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FPnUjsAMcXqV3uh7MX1l3%2Fuploads%2Fgit-blob-8c5c22974bd90eec6460a03db04c0192761c9964%2Fimage.png?alt=media)\\ # Golden Case CLAPNQ **Goal**\ Benchmarking to compare our retrieval system. Evaluating the retrieval part of our RAG System with a publicly available dataset that has ideal structured texts.\ \ **Data**\ Subset of the CLAPNQ Dataset. 104 questions, 724 chunks.\ Only documents with less than 10 chunks per documents were selected and a random sample was taken from them.\ Documents have between 1 and 10 chunks. The chunk containing the answer to the question is always included.\ \ **Method/Approach**\ Retrieval evaluated for three different vector embedding models:\ • Sbert\ • OpenAi text-embedding-3-small\ • OpenAI text-embedding-3-large\ \ **Results**\ The best performance measured by nDCG was 79% at k=5,6, while by CG a very high performance of 96% was achieved at k=6. Results were as expected given that nDCG is a stricter measure of quality. No large differences were observed between sBERT and OpenAI embedding models for this dataset.\ \ **Evaluation Metrics**\ Ranking quality: nDCG (Discounted Cumulative Gain), measuring the best possible result considering the right ranking.\ Retrieval quality: CG (Cumulative Gain), measuring total relevance, without penalizing for lower ranking.\ \ **Conclusions**\ The experiment produced expected results. Cumulative Gain (CG) reached a high of 96% at k=6, showing strong overall retrieval performance across all models. Although OpenAI’s embeddings (both small and large) showed marginally better ranking quality than sBERT, the difference was not significant enough to justify switching from our current sBERT model. # Comparative Retrieval Performance: Modules vs Golden Case **Goal**\ Compare CG from current modules with golden case\ \ **Data**\ Modules Kritisches Denken and Agiles Mindset, indexed xapi.\ 20 test questions for each module.\ \ **Method/Approach**\ Retrieval results evaluated using binary relevance score 1-0 (manually labeled relevant chunks).\ \ **Results**\ The best retrieval performance achieved for the module Kritisches Denken was 40% at k3. For Agiles Mindset the best achieved was 50% at k2.\ \ **Evaluation Metrics**\ Retrieval quality: Cumulative Gain (CG at k=1 to k=6)\ \ **Conclusions**\ The Cumulative Gain (CG) performance for the current modules was notably lower than the golden case. Further improvements are necessary to reduce the gap in our retrieval quality when compared to the ideal structured dataset of the golden case. *** ## Content retrieval details After testing with our real courses, we have decided to obtain a golden case of almost 5000 questions and answers and proceed with more experiments Experiments with this golden case and different LLMs do not show a big difference in the retrieved results\ ![](https://3619117356-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FPnUjsAMcXqV3uh7MX1l3%2Fuploads%2Fgit-blob-f5429f5006d2359dd56e34af4f9d35b3a1604a57%2Fimage.png?alt=media) But there is a big difference between the results from the golden case (above 90%) and our test learning content (that stays below 40% of retrieved results success) We decided to test different chunking strategies to try to improve the retrieval success scores. After using the sentence splitter strategy we got improved results for our test learning content:
# LLM-based Evaluator for Context Relevance **Goal**\ Assessing how well an LLM can identify relevant context chunks given question-text pairs so it can be used in a second-step of our retrieval system to filtered out irrelevant candidates.\ \ **Data**\ Modules Agiles Mindset, Kritisches Denken. 20 test questions from each module.\ \ **Method/Approach**\ LLM-based relevance scores from comparing question-chunk pairs. Prompt taken from TruLens Context Relevance.The LLM rates the context text chunk from 1 to 10. The score is normalized to a 0-1 scale.\ LLM used: gpt-4-0125-preview\ \ **Results**\ Recall of 100% in both Agiles Mindset and Kritisches Denken data.\ Precision: AM (71%) KD(33%).\ Accuracy: AM (81%) KD (52%)\ \ **Evaluation Metrics**\ Accuracy: The percentage of correctly predicted values.\ Recall: True positives / sum of true positives and false negatives (actual positives). Crucial when cost of false negatives is high. (predicted as 0 when it is in fact relevant)\ Precision: True positive/ sum of true positives and false positives (Total predicted positives '1').Crucial when cost of false positives is high. (predicted as 1 when it is not relevant).\ \ **Conclusions**\ The LLM-based context relevance evaluator was very good at correctly predicting all relevant chunks as relevant (100% recall). However, accuracy and precision was very different depending on the data evaluated. It was less accurate in general for the case of Kritisches Denken (52% accuracy) than for Agiles Mindset (71% accuracy). # Retrieval Performance Indexing pdf vs xapi, and Keywords vs Questions **Goal**\ Comparing retrieval performance across two indexing methods (pdf and xapi) and two query methods (Keywords vs Questions)\ \ **Data**\ Future Skills Module with 20 test questions.\ Keywords generated from questions using GPT-4 (gpt-4-1106-preview). Indexed documents in OpenSearch from pdf and xapi structure.\ \ **Method/Approach**\ Retrieval evaluation using two different indexing formats in OpenSearch (pdf and xapi) and two query methods (keywords vs full question). Vectoization with sbert. Relevance evaluated using gpt-4 as relevance grader (prompt based on trulens-eval library) to score similarity between query and retrieved chunks.\ \ **Results**\ Average relevance score for xapi index: 0.45. Average relevance score for pdf index: 0.3\ Full questions as queries resulted in a higher average relevance score (0.47) compared to keywords (0.34).\ \ **Evaluation Metrics**\ Mean Context Relevance Score (LLM-based): score from 0 to 1, averaged over 2 runs for each query.\ \ **Conclusions**\ Indexing based on xapi structure resulted in better retrieval relevance scores compared to pdf indexing.Using full questions as queries provides more accurate retrieval compared to keywords. So question-based retrieval is preferable for higher context relevance. # Install cluster ## Opensearch ### Getting started We are providing a customized [helm](https://helm.sh/docs/topics/charts/) script, in the directory `helm-deployment`, to allow easier deployment of [OpenSearch](https://opensearch.org/) in Kubernetes. First, create the namespace for the deployment, in this example `ctk-opensearch` is going to be used. #### 1. Deploying OpenSearch cluster Keep in mind OpenSearch can be deployed directly in AWS OpenSearch service using CloudFormation. The following instructions explain how to deploy OpenSearch in a Kubernetes cluster using helm charts: 1. Make sure the namespace is already created or create it with `kubectl create namespace ctk-opensearch` 2. Customize the **domains** from `ingress.host` within the script `opensearch.sh` (and make sure the DNS setup of those domains is pointing to the load balancer of the Kubernetes cluster) 3. Add OpenSearch helm repository `helm repo add opensearch https://opensearch-project.github.io/helm-charts/` 4. Enable `plugins.security.ssl.http` and **uncomment the lines with the certificates** in the section `opensearch.yml` from the file `opensearch-values.yaml` (ssl needs to be enabled in order to run the security script later on) Important values like `persistence.size` (the size of storage volumes) should also be customized from the file `opensearch-values.yaml` 5. **Run the script** `opensearch.sh` This should create pods `opensearch-cluster-master-x`. Testing the pods from within the same pods is easy with curl and the default credentials (admin admin): ``` curl -XGET http://localhost:9200 -u 'admin:admin' ``` Testing from anywhere in the internet should be also possible using the customized **domain** (possible with CURL or directly via the [browser](https://search.eduplex.eu)). ``` curl -XGET https://search.eduplex.eu -u 'admin:admin' ``` #### 2. Customize default admin password in static files (Optional) The first time the cluster is deployed, the default credentials should be changed. Future re-deployments on the same Kubernetes cluster will keep the same credentials, since this data is persistent even after namespace deletion. This **step is not needed** if you want to proceed with the password already stored in the file `.env.opensearch-edupl.env`. The hash in the file `helm-deployment/configMaps/internal_users.yml` (configured as **extraVolumeMounts** in the file `opensearch-values.yaml`) should be edited to change default passwords according to the [docs](https://opensearch.org/docs/latest/security/configuration/yaml/#internal_usersyml). The password stored in this file should be hashed. A [hash tool](https://github.com/opendistro-for-elasticsearch/security/blob/main/tools/hash.sh) is located in the container image. To run this tool, log-in into one of the pods from opensearch-cluster and run the script `/usr/share/opensearch/plugins/opensearch-security/tools/hash.sh -p aXr1x3k3VCqI5bu` Copy the hashed password from the output of the tool and update the hash `internal_users.yml` with this hash. The file `opensearch-dashboards-edu-secret.yaml` should also be updated including the new password, otherwise the connection between opensearch-dashboards (a.k.a Kibana) and the search endpoint will not work. 1. Change the password from file `.env.opensearch-edupl.env` 2. Run the command to generate the secret and seal it with [kubeseal](https://github.com/bitnami-labs/sealed-secrets). The command to generate the sealed secrets is available in the script file `opensearch.sh`, this command is starting with `#++` (because it does need to be executed only if the credentials need to be changed or a new Kubernetes cluster is used, an encrypted version of the secret file is stored in this repository) Totally delete all deployments (the easiest way to do this is by removing the namespace and creating it again). ``` kubectl delete namespace ctk-opensearch kubectl create namespace ctk-opensearch ``` Run `opensearch.sh` script again to **deploy OpenSearch**. #### 3. Deploying OpenSearch cluster with new password Wait a few minutes until OpenSearch is deployed Log-in into the pod. Optionally, check if the `internal_users` file was properly updated and using the correct hash. ``` head -n40 /usr/share/opensearch/config/opensearch-security/internal_users.yml ``` Then run the following script in order to update the passwords properly making them persistent (if this script is not executed, the password will not be updated): ``` /usr/share/opensearch/plugins/opensearch-security/tools/securityadmin.sh -cacert /usr/share/opensearch/config/root-ca.pem -cert /usr/share/opensearch/config/kirk.pem -key /usr/share/opensearch/config/kirk-key.pem -cd /usr/share/opensearch/config/opensearch-security/ ``` Once finished successfully, the connection can be tested using insecure https connection: ``` curl -XGET --insecure https://localhost:9200 -u 'admin:aXr1x3k3VCqI5bu' curl -XGET --insecure https://localhost:9200 -u 'admin:teoperro2023!' curl -XGET --insecure https://localhost:9200 -u 'admin:admin' curl -XGET http://localhost:9200 -u 'admin:aXr1x3k3VCqI5bu' curl -XGET http://localhost:9200 -u 'admin:teoperro2023!' curl -XGET http://localhost:9200 -u 'admin:admin' ``` The ssl certificate in production, will be handled by nginx-ingress controller and generated automatically using LetsEncrypt, for this reason, https inside the private network is not needed. Revert the changes in `plugins.security.ssl.http` in order to disable ssl certificate and **re-deploy OpenSearch** again using the script. After a few minutes, when the deployment is finished, testing the access via the [browser](https://search.eduplex.eu) or Curl using the newly changed **admin** account and **password** should be working. The connection with OpenSearch dashboards should be working properly on the configured domain. ### Create new users Following the [docs](https://opensearch.org/docs/latest/security/access-control/users-roles/#opensearch-dashboards), open OpenSearch Dashboards domain and go to Security > Internal Users and [Create internal user](https://adminsearch.eduplex.eu/app/security-dashboards-plugin#/users/create). Remember to select correct roles (for example a role to read only from index and another role to update documents to the index). To create role with index permisions in a certain index go to Management->Security->Roles->Create role Specify Index and index permissions
Create new user under Management->Security->Internal users
Once both user and role are created, assign the existing user to the role using the tab **Mapped users** in the role. # ESCO index ## ESCO data The full dataset in all languages was downloaded from [ESCO download page](https://esco.ec.europa.eu/en/use-esco/download) (as .csv) and included in this repository. **Maybe in some languages there's some file missing** The following files are located on the directory `esco/data/{lang_code}`. List of files in each language directory: * Skills: * **skillsHierarchy** (Define the hierarchy of the skills that has subskills, parent and children skills) * **skills** (Skills that don’t have children 'Final skills') * **broaderRelationsSkillPillar** (Define the skill and their parent skill) * **digCompSkillsCollection** (Collection of skills related to digComp) * **digitalSkillsCollection** (Collection of skills related to digitalSkills) * **greenSkillsCollection** (Collection of skills related to ecological skills) * **languageSkillsCollection** (Final language skills "***That they do not have children***" and their parents) * **researchSkillsCollection** (Collection of skills related to 'research' and the parents of them) * **skillGroups** (Skills that define a group or subgroup, like the skillsHierarchy) * **skillSkillRelations** (Relation between a skill with another skill, check optional Knowledge and optional for) * **transversalSkillsCollection** ('Final transversal skills' and their parents skills) * Occupations: * **broaderRelationsOccPillar** (Define the occupation and their parent occupation) * **ISCOGroups** (Define the hierarchy of the occupations, parent and children occupations) * **occupations** (All occupations) * **researchOccupationsCollection** (Occupations related to researching and their parents) * Other files: * **conceptSchemes** (Only in english language, Collection of schemas related to ESCO,ISCO,digComp) * **occupationSkillRelations** (Relation between occupations and skills that has the defined occupation) List of files needed to retrieve all skills & occupations: **Skills , occupations & occupationsSkillRelations** ## ESCO src In this directory we can find all the implementation needed to process the downloaded ESCO data from the `.csv` files in order to populate the ESCO indexes of `esco_occupations_sbert` and `esco_skills_sbert`. This implementation **ONLY** make use of the official languages of the EU, those languages are defined in a file within `esco/src/lib/data`. All the logic can be found on the executable file `escoIndexImporter.js` which is charge of the processing **ESCO data** to send it and store on the OpenSearch service. ### ESCO src/lib In this directory we can find subdirectories that are in charge of the configuration of the script #### ESCO src/lib/data In this directory we can find the javascript files in charge of configure the static values of the languages, language files and the index names. #### ESCO src/lib/service In this directory we can find the javascript files in charge of configure the HTTP Axios client and perform the HTTP request to the different API endpoints. *To perform HTTP requests to the **Sbert\_AI API** it is mandatory to first define an environment variable called: '**SBERT**'* with the value of the protocol, domain and port. Ex: '' ### [Run the implementation](#run-the-implementation) In order to run the script we recommend the use the **Node.js version 16.19.1** in order to avoid conflicts with **ESM modules** * To check the current version of node use the command: `nvm ls` * To install the recommended version use the command: `nvm install 16.19.1` * To change the current node version use the command: `nvm use ` * Run in local the `docker-compose` file of the [**Sbert\_AI**](https://gitlab.com/eduplex-api/text-analysis) repository * To perform HTTP requests to the **Sbert\_AI API** it is needed that the environment variable `SBERT` it is defined with the value of the protocol, domain and port. * To set the environment variable launch the next command on the CLI `export SBERT=http://localhost:5000` * To **run the script** it is only needed to run the command: `yarn start` over the directory `opensearch/esco` This command will install all the needed dependencies and run the `escoIndexImporter.js` file. ### Libraries needed To perform all the tasks to process and insert the data on the OpenSearch API the following libraries are needed: * [**Axios**](https://www.npmjs.com/package/axios): A promise based **HTTP** client that is in charge of sending and handling the HTTP request that are sended to the **OpenSearch API.** * [**CSV parser**](https://www.npmjs.com/package/csv-parser): A library that can convert **CSV** files into **JSON**. This library is needed to read the ESCO skills & occupations of each `.csv` file and parse into JSON to process the data easily. * [**Sbert\_AI API**](https://gitlab.com/eduplex-api/text-analysis): An AI API which vectorise input text like a skill description. ## Populate ESCO index In order to create and populate the indexes of `esco_occupations_sbert` and `esco_skills_sbert` with occupations and skills run the script `escoIndexImported.js` located in `esco/src` (Check the documentation in the section [Run the implementation](#run-the-implementation)). Once the script is executed the first step is that the script request to the user the credentials to allow to send HTTP to the OpenSearch API, if the username or password are not provided then the script execution will be finished. if the credentials provided are mistaken then the request to the OpenSearch API will return a **401 error code** and finish the execution. Then after the user introduce the credentials the script will read the data of the file `occupations_${langcode}` that is stored on `esco/data/` then if there is any **target language defined** on the function **`selectLanguages`** from the file `escoIndexImported.js` it is only read the data of those directories, otherwise all the data of the files are read and dumped on the machine memory (***The english language is always mandatory***). ### Esco occupations index Once the data of the files are dumped on the machine memory then it is first checked if any index with the name `esco_occupations_sbert` exist, if it not exists then it is created via **HTTP PUT** request with a configuration to create a **KNN OpenSearch index**, after that first step then it is processed all the data of `occupations_${langCode}` that is stored on `esco/data/`, **building an object**, for each **column of the file** a field is created in the object with the same name of the column and value of the current row of the file. Then it is added in each object a field called `langCode` with a 2 letter language code in order to identify from which language the occupation is, for the last step of the process the values of "*preferredLabel*", "*altLabels*" and "*hiddenLabels*" are merged to form a single string using a "\n" separator between each value and the description of each occupation are retrieved and send to the given **Sbert\_AI API** endpoint in order to get two vectors of **1024** dimensions one for the description and the other for the merged labels. Once the vectors are obtained they are added to a new fields called `vector_labels` and `vector_description` that are the same that the given in the configuration when creating the `esco_occupations_sbert` index. The task of vectorise all the occupations descriptions and labels from a language it takes a long time due that a lot of HTTP requests are sent one by one to the `Sbert_AI API`. This process could endure a long time. Finally, once the payload of each occupation is built, a batch of **200 occupations** is sent via **HTTP POST** request to the OpenSearch API in order to populate the `esco_occupations_sbert` index. The batch request are sent each **10s by default** and once **10 requests** are successfully sent there is a gap of **5 minutes by default** to give time to the index to perform the indexing operations (aka "Index Refresh") of the data that has received, otherwise **the index could be overwhelmed and close the socket rejecting the next requests** with a 429 error code or a timeout. ### Esco skills\_sbert index After the population of the `esco_occupations_sbert` index, then it is performed the same operations but with the skills, an index called `esco_skills_sbert` is created if not exist, and it is performed the same steps that the `esco_occupations_sbert` index did, the main difference with the `esco_occupations_sbert` index is that in the `esco_skills_sbert` index it is also read the file `occupationsSkillsRelations_${langCode}`that is stored on `esco/data/` in order to get the relations between skills and occupations, the data processing of each object add a field called `relatedOccupations` this field has the next structure: `[ { "occupationUri": "URI", "relationType": "relation", "occupationPreferredLabel": "label" } ]` Once the process of indexing finish there is **30m break**, in order to give time to the indexes to complete all unfinished indexing tasks and then an **HTTP POST** request is sent in order to merge the segments of each indexes into only one to achieve a better performance in the search operations of the indexes. **Sometimes this request takes a very long time to perform the merge operations, and it could automaticallt close the socket connection after a while. This doesn't mean that the operation fails, only that the response wasn't available after a few time.** # Learning opportunities index Learning opportunities, services or courses are the basic entity for learning. ## Creating index To create an index PUT to `https://search.eduplex.eu/example` being `example` the index name. You can add the mapping fields in the body. See an example here: `https://gitlab.com/eduplex-api/opensearch/-/blob/main/files/index_mapping.json` ## Retrieving entries from index To search on the index you can do a GET to `https://search.eduplex.eu/example/_search` being `example` the index name and with a body like this it will return all documents from the index. ``` { "query": { "match_all": {} } } ``` We also have a proxy on our API to build more complex queries allowing to filter, return facets or rank results. Our endpoint allows the following parameters: * **page**: pagination number. * **limit**: number of documents per page. * **text**: text to search. * **semantic\_text**: same as previous parameter but using vector search with exact knn. You can not use both at the same time. * **sort\_by**: it allows to sort by the following fields: * **title** * **total\_price** * **ratings** * **next\_course** * **order**: it changes the sorting criteria (`asc` or `desc`) and the following filters: * **facets**: an object with different facets to filter. E.g.`{ type: 'online', city: 'Viena' }`. * **tag**: returns only events associated to a specific tag. * **category**: returns only events associated to a specific category. * **seller\_id**: filters by the provider of the learning opportunities. * **is\_internal**: returns only internal events. * **has\_exclusive\_tag**: if `false` returns only events without exclusive tags. * **exclusive\_tags**: if previous filter is set to `false`, also returns events with the specified tags. ## Examples We use some special parameters to modify the results: * **track\_scores**: it forces to calculate the score of the documents. * **functions**: it allows to customize documents ranking scores. * **score\_mode**: how to calculate the result from all the functions. * **boost\_mode**: how to apply the `score_mode` to the document score. * **aggs**: facets to be returned. ### Text search query ``` GET example_index/_search { "from": 0, "size": 10, "track_scores": true, "query": { "function_score": { "score_mode": "sum", "boost_mode": "multiply", "query": { "bool": { "must": [ { "multi_match": { "query": "", "fields": [ "title^2", "highlight", "tags.display_name", "categories.display_name", "module_titles" ], "fuzziness": "AUTO", "zero_terms_query": "all" } } ] } }, "functions": [ { "filter": { "term": { "provider_is_verified": true } }, "weight": 1.2 }, { "filter": { "term": { "provider_is_topseller": true } }, "weight": 1.2 }, { "filter": { "term": { "is_bookable": true } }, "weight": 7 }, { "filter": { "term": { "has_boost": true } }, "weight": 7 }, { "field_value_factor": { "field": "rating_rounded", "factor": 0.01, "missing": 250 } } ] } }, "aggs": { "tags.display_name": { "terms": { "field": "tags.display_name.keyword", "size": 100 } }, "language": { "terms": { "field": "language.keyword", "size": 10 } }, "categories.category": { "terms": { "field": "categories.category.keyword", "size": 100 } }, "provider.name": { "terms": { "field": "provider.name.keyword", "size": 100 } } } } ``` ### Semantic search query ``` GET example_index/_search { "from": 0, "size": 10, "track_scores": true, "query": { "function_score": { "score_mode": "sum", "boost_mode": "multiply", "query": { "bool": { "must": [ { "bool": { "should": [ { "script_score": { "query": { "match_all": {} }, "script": { "source": "knn_score", "lang": "knn", "params": { "field": "highlight_vector", "query_value": [], "space_type": "innerproduct" } } } } ] } } ] } }, "functions": [ { "filter": { "term": { "provider_is_verified": true } }, "weight": 1.2 }, { "filter": { "term": { "provider_is_topseller": true } }, "weight": 1.2 }, { "filter": { "term": { "is_bookable": true } }, "weight": 7 }, { "filter": { "term": { "has_boost": true } }, "weight": 7 }, { "field_value_factor": { "field": "rating_rounded", "factor": 0.01, "missing": 250 } } ] } }, "aggs": { "tags.display_name": { "terms": { "field": "tags.display_name.keyword", "size": 100 } }, "language": { "terms": { "field": "language.keyword", "size": 10 } }, "categories.category": { "terms": { "field": "categories.category.keyword", "size": 100 } }, "provider.name": { "terms": { "field": "provider.name.keyword", "size": 100 } } } } ``` # Search test cases This page will contain a list of different inputs to perform search expecting good results ## Esco occupations by plain text (DE) Search terms tested: * web * webdesign * design * webmaster ~~webmas~~ * proj, projekt, projektmana ## Esco skills by plain text (DE) Search terms tested `https://proto.eduplex.eu/edu/api/v1/openSearch/esco_skills?plain_text=`: * microsoft * physiotherapiedienstleistung * physiotherap * physio ## Esco skills by occupation vector search (DE) Using occupation labels, we get all the skills with their description vectors and perform vector search Occupation labels tested: * Web-Designer/Web-Designerin,Webmaster/Webmasterin # Abstract The **Skill Assessment Builder** (Module Occupation Matching), is a comprehensive full-stack system designed to facilitate the creation and management of quizzes for skill evaluation. It enables the generation of detailed user profiles and provides deep insights into users' knowledge and competencies. The system supports both extensive and simple quizzes, offering flexibility to cater to diverse assessment needs. Fully functional and successfully tested in the prototype, this module serves as a powerful tool for personalized skill assessment and occupation matching, enhancing user understanding and engagement. # Module Occupation Matching ## ASSESSMENT frontend Competency analysis tool (KYC) ## General info #### Global dependencies: * Docker and [docker-compose](https://docs.docker.com/compose/) Use docker images or run with node.js natively * [Node.js](https://nodejs.org/en/) Recommended to install using [NVM](https://github.com/nvm-sh/nvm) and running `nvm install` * [yarn](https://classic.yarnpkg.com/en/docs/install/#debian-stable) (package manager used instead of npm) * **git** as version control system using [rebase](https://git-scm.com/book/en/v2/Git-Branching-Rebasing) instead of merge. * The backend API will be available in the repository [MOM assessment backend](https://gitlab.com/eduplex-api/mom-assessment-frontend) #### Frameworks: * [VueJS](https://vuejs.org/) main javascript framework * [Vuetify](https://vuetifyjs.com/en/introduction/why-vuetify/) UI framework with handy components * [TypeScript](https://www.typescriptlang.org/) As programming language #### Environment variables: * `VUE_APP_API` Location for the API service * `VUE_APP_SUBPATH` Subdomain where the app is going to be served * `VUE_APP_PLATFORM` To be used as platform identifier (currently not used) * `VUE_APP_TARGET_PROXY` Real domain to connect to the backend API after proxy forward * `DATE_IMG_TAG` Version number #### Platform customizations: * **Configuration endpoint** `/api/v2/configs/mom_front` should return from database a list of entries to allow platform configuration * Custom **themes** can be added in the config endpoint using the key `color_theme`, adding the files to `/pluings/themes/.ts` and importing them into `const themesIndex` from `/pluings/themes/index.ts` * Custom **translations** can be added in the config endpoint using the key `custom_locale` and adding the files to `/pluings/i18n/-.ts` ## Build Setup Useful notes when running development built: * API requests are been sent via a proxy forward defined with `VUE_APP_TARGET_PROXY` to avoid CORS issues. * Login action in production is different that in development * Logout action is not implemented in development (in order to logout, cookies have to be removed manually) * Related links are going to work different (while in development they are going to look like localhost:8080, when deployed to production those relative links are going to change to the real production domain) #### With Docker: [Docker compose](https://docs.docker.com/compose/) file is ready to use, just run `docker-compose -f ./docker-compose.yml up -d`. When working with `docker-compose.yml` it is important to change the environment variable `VUE_APP_TARGET_PROXY` to the correct domain where the API endpoints are available. You can access the login page by default going to where `assess` is defined with environment variable `VUE_APP_SUBPATH`. #### Without Docker: Alternatively you can work without docker with the following commands: Install dependencies: ```bash yarn install ``` Build and hot-reload for development: ```bash yarn dev ``` Build and minify for production: ```bash yarn build ``` Lint: ```bash yarn lint ``` ### Customize configuration See [Configuration Reference](https://cli.vuejs.org/config/). ## License The source code for the site is licensed under the **MIT license**, which you can find in the LICENSE file included in this repository. # Skill Assessment tracking with LRS Currently we are tracking **question answers** and **initiatives scores**. * Every time a user goes through a questionnaire we are sending statements for each question he answers.
Click to see an example ``` { "stored": "2023-07-12T14:11:03.286Z", "active": true, "completedForwardingQueue": [], "failedForwardingLog": [], "client": "62ed2d33caa1070008fbe17d", "lrs_id": "62ed2d33caa1070008fbe17c", "completedQueues": [], "activities": [ "https://proto.eduplex.eu/questionnaire/4/user/3/question/173/answer/758" ], "hash": "6a7f5bbcdc5e36eb680d47a8eb23d857b44015a5", "agents": [ "https://proto.eduplex.eu|3" ], "statement": { "authority": { "objectType": "Agent", "name": "EduPLEx Client", "mbox": "mailto:adrian@courseticket.com" }, "stored": "2023-07-12T14:11:03.286Z", "context": { "extensions": { "https://proto.eduplex.eu/xapi/extension/user_answers": { "user_id": 3, "manager_id": null, "questionnaire_id": 4, "question_id": 173, "question_text": "ihr Computer zeigt seltsame Störungen. Was hilft in vielen Fällen?", "answer_id": 758, "answer_text": "Ein Neustart des PCs.", "start_date": "2023-07-12 16:10:29" } } }, "actor": { "account": { "homePage": "https://proto.eduplex.eu", "name": "3" }, "objectType": "Agent" }, "timestamp": "2023-07-12T14:11:03.286Z", "version": "1.0.0", "id": "7402ec62-65a2-4875-acb3-d538ef50911d", "verb": { "display": { "en-US": "answered" }, "id": "http://adlnet.gov/expapi/verbs/answered" }, "object": { "definition": { "type": "https://proto.eduplex.eu/xapi/object/types/user_answers" }, "id": "https://proto.eduplex.eu/questionnaire/4/user/3/question/173/answer/758", "objectType": "Activity" } }, "hasGeneratedId": true, "deadForwardingQueue": [], "voided": false, "verbs": [ "http://adlnet.gov/expapi/verbs/answered" ], "processingQueues": [], "person": null, "timestamp": "2023-07-12T14:11:03.286Z", "relatedActivities": [ "https://proto.eduplex.eu/questionnaire/4/user/3/question/173/answer/758" ], "relatedAgents": [ "https://proto.eduplex.eu|3", "mailto:adrian@courseticket.com" ], "organisation": "62ed1d89858c0c0008338e5c", "_id": "64aeb4776d0a00001bf75e49", "registrations": [], "pendingForwardingQueue": [] } ```
* After the user completes the questionnaire we are sending one statement for each initiative with the score.
Click to see an example ``` { "stored": "2023-07-12T14:11:03.864Z", "active": true, "completedForwardingQueue": [], "failedForwardingLog": [], "client": "62ed2d33caa1070008fbe17d", "lrs_id": "62ed2d33caa1070008fbe17c", "completedQueues": [], "activities": [ "https://proto.eduplex.eu/assessment/Erstellung digitaler Inhalte" ], "hash": "92256d68d28d0b6f16cbfacfd5be330002650703", "agents": [ "https://proto.eduplex.eu|3" ], "statement": { "authority": { "objectType": "Agent", "name": "EduPLEx Client", "mbox": "mailto:adrian@courseticket.com" }, "stored": "2023-07-12T14:11:03.864Z", "actor": { "account": { "homePage": "https://proto.eduplex.eu", "name": "3" }, "objectType": "Agent" }, "timestamp": "2023-07-12T14:11:03.864Z", "version": "1.0.0", "id": "ccc49562-e8b6-49af-a36e-c4eae89868d7", "result": { "score": { "raw": 56, "min": 0, "max": 100 } }, "verb": { "display": { "en-US": "scored" }, "id": "http://adlnet.gov/expapi/verbs/scored" }, "object": { "definition": { "type": "http://adlnet.gov/expapi/activities/assessment", "name": { "en-US": "Erstellung digitaler Inhalte" } }, "id": "https://proto.eduplex.eu/assessment/Erstellung digitaler Inhalte", "objectType": "Activity" } }, "hasGeneratedId": true, "deadForwardingQueue": [], "voided": false, "verbs": [ "http://adlnet.gov/expapi/verbs/scored" ], "processingQueues": [], "person": null, "timestamp": "2023-07-12T14:11:03.864Z", "relatedActivities": [ "https://proto.eduplex.eu/assessment/Erstellung digitaler Inhalte" ], "relatedAgents": [ "https://proto.eduplex.eu|3", "mailto:adrian@courseticket.com" ], "organisation": "62ed1d89858c0c0008338e5c", "_id": "64aeb4776d0a00001bf75e4a", "registrations": [], "pendingForwardingQueue": [] } ```
We are also displaying a list of results after the user completes the questionnaire:
# LTI Connector [LTI](https://www.imsglobal.org/spec/lti/v1p3#platforms-and-tools) implementation ## Works with CakePHP Plugin [repository](https://gitlab.com/eduplex-api/cake-api-lti) to run on top of [cake-rest-api](https://packagist.org/packages/freefri/cake-rest-api). ## Openapi documentation Swagger UI in [/api/v3/lti/openapi/](https://proto.eduplex.eu/api/v3/lti/openapi/) ## Demo Tool workflow Here is a flowchart of the launch process: {% @mermaid/diagram content="sequenceDiagram title LTI demo launch flow participant User as User participant Platform as LTI Platform
(Tool Consumer) participant Tool as LTI Tool
(Tool Provider) User->>Platform: Access Tool via iframe Platform->>Tool: Render iframe in loginUrl Note right of Tool: Params:
iss
lti\_deployment\_id
client\_id
target\_link\_uri (redirect URI) Tool->>Tool: do\_oidc\_login\_redirect() Tool->>Platform: Redirect to Platform loginUrl(client\_id, redirect\_uri) Note left of Platform: Params:
scope
response\_type
response\_mode
prompt
client\_id
redirect\_uri
state
nonce Platform->>Tool: Post form to redirect\_uri Note right of Tool: Params:
state
id\_token (JWT) Tool->>Tool: Validate JWT (LtiMessageLaunch)" %} ## License The source code for the site is licensed under the [**MIT license**](https://gitlab.com/eduplex-api), which you can find in the [LICENSE](https://gitlab.com/eduplex-api/cake-api-lti/-/blob/main/LICENSE/README.md) file. # LTI setup Some settings need to be exchanged manually between Platform and Tool. Settings provided from the tool: * Tool URL: * Initiate login URL `Platform.LTI.toolLoginUrl`: * Redirection URL(S) `Platform.LTI.toolRedirectionUrl`: * Public Keyset URL `Platform.LTI.toolKeysetUrl`: Settings provided from the Platform: * Platform ID/URL: * Client ID `Platform.LTI.clientId`: xxxxxxxx-yyyy-xxxx-yyyy-xxxxxxxxxxxx * Deployment ID `Platform.LTI.deploymentId`: xxxxxxxx-yyyy-xxxx-yyyy-xxxxxxxxxxxx * Public keyset URL * Access token URL * Authentication request URL In the Platform Config settings, we should store the config values, to be accessible via `Configure::read('Platform.LTI.<…>')`. An RSA private key must be generated and stored in the .env `RSA_PRIVATE_KEY_BASE64`: ``` openssl genrsa -out private.key 2048 openssl rsa -in private.key -pubout -out public.pem echo "RSA_PRIVATE_KEY_BASE64=" && base64 -w 9999 private.key ``` # Verwertungsplan ### **Verwertungsplan nach Projektende** #### **1. Öffentlichkeitsarbeit** Nach Projektende sind bereits weitere Teilnahmen an Messen und Kongressen geplant, u.a. wird eine Teilnahme als Aussteller und Keynote-Speaker am HR Expo Festival in Wien im Februar 2025 angestrebt. Am Standort Köln / über KölnBusiness ist eine weitere Verbreitung der Projektergebnisse im Rahmen von Edtech-Meetups und Messen (Digitaltag Kölner Unternehmen) in Planung.\ Die Projektpartner streben eine Bewerbung über ihre eigenen offiziellen Webauftritte und Social Media Kanäle an. Courseticket verfolgt das Ziel, eigene Webauftritte für AI-Tutor, AI-Assistent und AI-Fellow zu entwickeln, sodass effiziente Marketingkampagnen (Search Engine Marketing) betrieben werden können. #### **2. Nachnutzung durch Dritte** Mit der TU München (Thinktank) sowie der FH Wien (Teaching & Learning Center) wird ein Plattform-Einsatz und die gemeinsame Weiterentwicklung für unterschiedliche Einsatzgebiete forciert.\ Die öffentlich erreichbaren Projektergebnisse (GitBook-Dokumentation, GitLab-Repositories) sollen kontinuierlich ergänzt und erweitert werden. Den entsprechenden Ausgangspunkt bilden dabei die zentralen Projekt-Dokumentationsplattformen, die unter GitBook () erreichbar sind. #### **3. Öffnung gegenüber Zielgruppe mit besonderen Bedürfnissen** Über geplante Folgeanträge der WBS TRAINING AG und Courseticket (u.a. “Wandel der Arbeit”, Ausschreibung des BMAS) soll die KI-gestützte Learning Experience in eine barrierefreie “Inklusive Learning Experience” übergeführt werden.\ Hierbei wird eine Zielgruppe (Einschränkung durch Behinderung) erreicht, der es bis dato verwehrt blieb, von künstlicher Intelligenz im selbstgesteuerten, personalisierten Lernen zu profitieren. #### **4. Praktische Erprobung von digitalen Bildungsnachweisen** Im Zuge eines Antrags auf EU-Ebene (“Cooperation partnerships in higher education”) soll in Zusammenarbeit mit mehreren Hochschulen eine Infrastruktur zum Austausch digitaler Zertifikate und Bildungsnachweise aufgebaut werden.\ Durch eine enge Abstimmung zwischen den Projektpartnern aus Hochschulbereich (HEI), Vocational Training (VET) und Technologiedienstleister wird ein nahtloser Ausstellungs- und Verifizierungsprozess von Verifiable Credentials (Digital Diplomas) angestrebt. #### **5. Iterative Implementierung des Napil°AI-Systems in Lernangebote der WBS TRAINING AG** Das Napil°AI-System wird schrittweise in die bestehenden und zukünftigen Lernangebote der WBS TRAINING AG integriert. Die Einführung erfolgt iterativ auf Basis gezielter Datenauswertung und Rückmeldeschleifen. Ziel ist es, datenbasierte Erkenntnisse systematisch zu nutzen, um den didaktischen und technischen Einsatz weiter zu optimieren und die Skalierbarkeit des Systems in der Breite zu gewährleisten. #### **6. Integration der Erkenntnisse aus dem AI Companion in die Lernunterstützung** Die im Projektverlauf gewonnenen Erkenntnisse aus der Entwicklung und Nutzung des AI Companion fließen direkt in die lernbegleitenden Unterstützungsangebote der WBS TRAINING AG ein. Dabei wird besonderer Wert auf adaptive Lernpfade, individualisierte Hilfestellungen und dialogorientiertes Feedback gelegt, um Lernende bedarfsgerecht und effizient zu fördern. #### **7. Einsatz von Business Action Games als ergänzendes Lernformat** Business Action Games werden als niedrigschwelliges, spielerisches Zusatzformat in das Weiterbildungsportfolio der WBS TRAINING AG integriert. Diese Formate dienen der Anwendung und Vertiefung von Kompetenzen in simulierten Entscheidungssituationen und fördern den Transfer in reale berufliche Kontexte. Erste konzeptionelle Vorarbeiten sind abgeschlossen; eine Pilotierung ist geplant. #### **8. Integration trainergestützter Lernszenarien in Produkte** Die entwickelten Konzepte für trainerunterstützte Lernszenarien werden zukünftig als feste Bestandteile in die Produktentwicklung der WBS TRAINING AG aufgenommen. Ziel ist es, die Kombination aus KI-gestütztem Lernen und persönlicher Trainerbegleitung als hybrides Format nutzbar zu machen. Diese Szenarien stärken insbesondere die soziale Interaktion, fördern kollaboratives Lernen und ermöglichen eine engmaschige Begleitung bei komplexen Lerninhalten**.** #### 9. Weiterer Praxiseinsatz des digitalen Lehrformats Das digitale Lehrformat wird derzeit in vier Kursen an der Hochschule Schmalkalden eingesetzt. Ab Sommersemester 2025 wurden weitere Hochschulen als Anwender kontaktiert. Die daraus resultierenden Gespräche deuten darauf hin, dass das digitale Lehrformat zumindest testweise im akademischen Bereich an weiteren Hochschulen eingesetzt werden wird. Darüber hinaus ist vorgesehen eine Kooperation mit der Fa. acemate.ai einzugehen, um das digitale Lehrformat mit den Werkzeugen von acemate zu ergänzen und später in der Aus- und Weiterbildung bei KMUs einzusetzen. # Requirement-Paper - offene technologische Entwicklungen ### AP1 – User behavior tracking User behavior is being successfully tracked at the **individual level** using xAPI, allowing for detailed insights into personal learning interactions. This supports personalized feedback and analytics aligned with the project goals. However, tracking **groups of users** is no longer feasible due to challenges in grouping heterogeneous user profiles, lack of a consistent grouping model, and concerns related to privacy and data ethics. As a result, group-level behavior tracking has been excluded from the current implementation. ### AP4 – Configuration / Extension of xAPI / LRS (Statements, Queries, Activities, Metadata) The current work on AP4 has focused on enhancing the visualization of learning activity data collected through xAPI. Several reports of xAPI statements were implemented with different charts, providing an initial overview of user activity, learning progress, and competence areas. Although the initial idea was to implement some of these visualizations directly within the LRS, they were ultimately integrated into the LXP prototype instead. This decision was made to offer a more user-friendly interface and facilitate easier access and interaction for end users. The implementation of a complete visualization of the user journey was initiated, aiming to map out learners’ paths through activities and interactions within the LXP. However, initial testing revealed significant challenges in determining which data points were most relevant and how to present them in a meaningful and user-friendly way. Due to the complexity and ambiguity around visualization approaches, and considering project priorities, this feature was deprioritized. As a result, the development of a fully functional and usable version of the user journey visualization was halted. ### AP6 - Implementierung Workflow Anpassungen (Statements/Meta Schemata) For AP6, the implementation of workflow adjustments included extending xAPI statements by adding schema.org metadata to enrich the behavior tracking data. Although this enhanced data is stored within the system, it has not been actively utilized in the project so far. Initial plans also included integrating with the Nationale-Bildungsplattform (NBP) and the Allgemeines Metadatenprofil für Bildungsressourcen (AMB), which is derived from LOMS standards. However, during the course of the project, it was decided that these integrations were not essential, and no further development was pursued in this area. ### AP8 - Design & Implementation Semantic Search & On-site Search API For AP8, the design and implementation of semantic search and on-site search APIs focused on integrating OpenSearch with embeddings and k-nearest neighbors (kNN) search for ESCO skills and learning opportunities. This setup enables the LXP to identify semantically similar skills and opportunities, forming the foundation for the recommendation and user engagement engines. Although the initial plan included using embedding-based semantic search for learning opportunities, early tests showed that non-semantic search generally produced better results. Consequently, semantic search was not adopted as a full replacement across the LXP and was integrated only in some autocomplete components. ### AP9 - Predictive Analitics Dashboard In AP9, the Predictive Analytics Dashboard was implemented as a user interface component within the LXP prototype, designed to support learners in identifying skill gaps and receiving personalized learning recommendations. The dashboard provides visual insights based on user activity and profile data, helping individuals understand their progress and make informed decisions about what to learn next. Although the original plan included advanced predictive analytics tools for LXP operators and content creators, time constraints and technical complexity led to a shift in focus towards end-user functionality. Nonetheless, several analytics charts and visual elements have been made available on the operator side to support monitoring and decision-making at a higher level. ### AP11 - User Engagement Engine In AP11, the original goal was to set up triggers to boost user engagement through real-time notifications about recommendations, new learning content, and comparative performance data (e.g., performing better than average). Additionally, the plan included developing an engagement engine with a user-friendly interface to enable AI-driven interactions with learners. Instead of the initial approach, we implemented automated H5P quizzes with ranking features to foster engagement. Later, this was expanded by integrating a Retrieval Augmented Generation system based on a large language model (LLM) AI tutor and assistant. However, due to time constraints, real-time integration of xAPI performance data with the AI tutor is still pending. # Requirement-Paper - Allgemeine Aspekte ### Empfehlungen und zu berücksichtigende Aspekte nach Projektabschluss Im Rahmen des abgeschlossenen Forschungsprojekts wurden zentrale technologische, regulatorische und didaktische Entwicklungen identifiziert, die potenziell Einfluss auf die Weiterentwicklung und nachhaltige Nutzung der entwickelten Lösung haben. Im Folgenden sind die wesentlichen Punkte aufgeführt, deren Berücksichtigung für zukünftige Maßnahmen relevant ist: #### 1. Datenschutz und Regulierung * **Einhaltung der DSGVO**\ Künftige Entwicklungen sollten im Einklang mit der Datenschutz-Grundverordnung (DSGVO) stehen. Besonders im Fokus stehen hierbei Datenminimierung, Zweckbindung sowie transparente Datenverarbeitung. * **Berücksichtigung des EU AI Acts**\ Da die Plattform als potenzielles Hochrisiko-KI-System einzustufen ist, sind Anforderungen aus dem EU AI Act antizipierend zu berücksichtigen, etwa in Bezug auf Dokumentation, Erklärbarkeit und Risikobewertung. * **Verzicht auf problematische Gruppierungen (Personas)**\ Die Aggregation von Nutzerdaten zu konkreten Merkmalen (und somit die Erstellung von "Personas") ist kritisch zu hinterfragen. Es gilt, Risiken einer Re-Identifikation zu vermeiden und die rechtliche Zulässigkeit sorgfältig zu prüfen. #### 2. Interoperabilität und Standardisierung * **Erweiterung der ESCO-Taxonomie**\ Obwohl die Plattform bereits ESCO-Skills zur Kompetenzstrukturierung verwendet, besteht künftig Bedarf an fachspezifischen Ergänzungen, um domänenspezifische Anforderungen abzudecken. * **xAPI-Kompatibilität sicherstellen**\ Der Einsatz der xAPI als Standard ist beizubehalten, wobei auf eine einheitlichere und robustere Datenstruktur bei xAPI-Statements in kommenden Versionen zu achten ist. #### 3. KI-Nutzung und Personalisierung * **Bevorzugung von Open-Source-LLMs**\ Vor dem Hintergrund des Datenschutzes und der besseren Kontrollierbarkeit von Datenflüssen erscheinen Open-Source-Sprachmodelle perspektivisch vorteilhaft. * **Weiterentwicklung der RAG-Architektur**\ Die bestehende Infrastruktur auf Basis von Retrieval Augmented Generation (RAG) bildet das Fundament personalisierter Empfehlungen und sollte weiterentwickelt werden, um kontextsensitive Interaktionen zu ermöglichen. * **Optimierung der Prompt-Datenbank**\ Die Steuerung des AI-Companions erfolgt über strukturierte Prompts. Diese sollten fortlaufend gepflegt, erweitert und qualitativ optimiert werden (Stichwort: Prompt Engineering). #### 4. Didaktische Architektur (NAPIL°AI) * **Automatisierung bedarfsgerechter Lernpfade**\ Die Fähigkeit zur Generierung individueller Lernlösungen im „Moment of Need“ erfordert eine weitere Verfeinerung der Algorithmen zur Inhaltsaufteilung (Chunking) und zum semantischen Matching. * **Stärkung der didaktischen Steuerung**\ Das NAPIL°AI-System fungiert als didaktisches Steuerungsinstrument. Es wird empfohlen, zusätzliche relevante Datenpunkte für die Lernpfadsteuerung iterativ einzubeziehen. #### 5. Analyse- und Bewertungssysteme * **Ausbau von Predictive Analytics**\ Für eine effektive Erkennung von Lernmustern und zur Bewertung von Kompetenzentwicklungen sind leistungsfähige Datenmodelle und unterstützende Visualisierungstools (z. B. User Journey Visualizer) erforderlich. * **Qualitätssicherung der KI-Ergebnisse**\ Die Qualität der KI-Antworten hängt maßgeblich von der verwendeten Einbettungstechnologie (Embeddings), der Prompt-Strategie sowie der zugrundeliegenden Datenbasis ab. Diese Komponenten sollten kontinuierlich optimiert werden. * **Systematische Auswertung von KI-Interaktionen**\ Die aus KI-gestützten Nutzerinteraktionen gewonnenen Daten bergen hohes Potenzial für Learning Analytics. Eine strukturierte Auswertung könnte zur Qualitätssicherung und Weiterentwicklung beitragen. ### Didaktische Erweiterung des digitalen Lehrformats (Hochschule Schmalkalden) **Ziel des digitalen Lehrformats von EduPLEx\_API ist der Erwerb konkreter akademischer und beruflicher Handlungskompetenzen, die sich direkt im betrieblichen Alltag anwenden lassen**. Besonders in diesem Kontext sind digitale Lernformate gefragt,\ die selbstbestimmte Lernen in kontrollierten Lernumgebungen unterstützen (**Lernaspekt**)\ und die das Angebot sehr spezifischer und dynamisch wechselnder Lehrinhalte ermöglichen (**AutorInnenaspekt**) Beide Aspekte sollen in Zukunft in unserem digitalen Lehrformat umgesetzt werden, da beide derzeit von keiner Plattform für digitales Lernen in ausreichendem oder gar zufrieden stellendem Maße abgedeckt werden.\ Unsere Erfahrungen aus EduPLEx\_API zeigen, dass die vielfältigen Facetten des Lernens von digitalen Instrumenten derzeit nur sehr punktuell und fragmentarisch adressiert werden. Das Lernen in seiner Komplexität und Vielseitigkeit zu unterstützen reicht weit darüber hinaus, was KI-gestützte und sprachorientierte Instrumente (wie ChatGPT-basierte Systeme) heute vermögen. Parameter wie Vergessensrate, Retention Rate, Identifikation produktiver Lernphasen etc. sind zwar theoretisch sehr gut beschrieben, aber in digitalen Lernformaten überhaupt nicht berücksichtigt. Die KI, insbesondere das Maschinelle Lernen, kann Instrumente liefern Lernende zu beobachten und Verhaltensparameter festzustellen, aus den Gestaltungsempfehlungen für digitale Lernformate abgeleitet werden können. **Die Herausforderung für das digitale Lehrformat das Verhalten der Lernenden in seine Technologiegestaltung einzubeziehen bietet Chancen aber auch Risiken.** In jedem Fall kann der Herausforderung nur über die Gestaltung einer neuen Technologie für das Lernen begegnet werden. Aufgabe der KI ist es in diesem Kontext Sprachmodelle mit statistischen Modellen, die das Verhalten (datenschutzrechtskonform) abbilden, so zu verknüpfen, damit Lernen nachhaltig unterstützt wird. Potentiale und Risiken werden sich erst im Rahmen der Technologiegestaltung umfänglich substanziieren. #### 1.1 Lernaspekt Unser digitales Lehrformat soll auch die **Generierung von Kursen** unterstützen, **die von den Lernenden autonom erlernt werden können**. Ganz egal welche Inhalte erlernt werden sollen, es ist wichtig, dass die Lernenden sie erlernen können, wo und wann immer sich ein geeigneter Ort und ein geeigneter Zeitpunkt für ein effizientes und zielführendes Lernen ergibt. Neben der zeitlichen und inhaltlichen Flexibilität soll das digitale Lehrformat auch unterschiedliche Lernpräferenzen unterstützen, z. B. visuelles, auditives oder handlungsorientiertes Lernen, um eine möglichst effektive und motivierende Lernumgebung zu schaffen. Diese Rahmenbedingung schließt viele pädagogische Aspekte ein, die in den KI-Modellen für digitale Lernplattformen berücksichtigt werden müssen und die weit über den Bereich der sprach- oder LLM-gestützten KI hinausgreifen. Wir müssen uns darüber im Klaren sein, dass Verhaltensaspekte des Lernens wie individuelles Zeitmanagement, Reflexionsimpulse, Leistungseinschätzung usw. in Lernplattformen für selbstbestimmtes Lernen wenn überhaupt in nur sehr rudimentärer Form berücksichtigt werden.\ *Selbstbestimmtes Lernen kann nur dann erfolgreich sein, wenn es in kontrollierten Umgebungen stattfindet, in Umgebungen, die die Anwesenheit einer Tutorin oder eines Tutors so gut wie möglich simulieren können*. Diese **tutorielle Begleitung** manifestiert sich in adaptiven Lernpfaden, gezielten Rückfragen, motivierenden Rückmeldungen und der dynamischen Anpassung von Inhalten an den Lernfortschritt und den individuellen Lernpfads. Es ist alles andere als trivial, einem Menschen den individuellen Lernpfad zu vergegenwärtigen, insbesondere dann, wenn Lerninhalte und Lernziele ihm noch unbekannt sind. Diese Ausgangssituation für das Lernen ist typisch für die innerbetriebliche Aus- und Weiterbildung, vor allem bei KMUs. Selbst wenn die Themen überbetrieblichen, fach- oder industriespezifischen Bezug haben, so müssen sie meistens mit spezifischen Realitäten gekoppelt werden, um die nötige praxisnahe Aneignung von neuem Wissen zu ermöglichen. Hier geht es um spezifisches Prozess-, Vorgangs- und Fachwissen, das im betrieblichen Kontext erst komplettiert und zur Anwendung kommt. Das akademische Lernen unterscheidet sich davon deutlich. Studierenden sind Umfang und Struktur des zu erlernenden Wissensgebiets zumindest soweit vertraut, dass sie selbstständig (und mit KI-Unterstützung) Lehrmaterial zusammenstellen und sich daraus auch KI-gestützt Lernkontrollen erstellen lassen können. Sie kümmern sich selbständig um ihren Lernpfad, ihr Zeitmanagement und alle weiteren Aspekte, die erfolgreiches Lernen auszeichnen. Zumindest sollten sie das können. In einem betrieblichen Setting für die Aus- und Weiterbildung kann man nicht davon ausgehen, dass die MitarbeiterInnen diese Kultur des autonomen Lernens mitbringen. Das gilt auch in einigen Bereichen der akademischen Ausbildung von Studierenden, insbesondere in den Anfangsphasen des Studiums. Hier muss das digitale Lernformat in der Lage sein, Lernende an der Hand zu nehmen und sie auf ihrem Lernpfad zu begleiten. **Diese pädagogisch-didaktische Begleitung wird derzeit von keiner digitalen Lernplattform angeboten**. Es geht dabei nicht nur um die Verknüpfung von mächtigen Sprachmodellen mit Lerninhalten, es geht auch und vor allem um die Gestaltung und Integration KI-gestützter Modelle, die Verhaltensaspekte des Lernens abdecken, um wirklich selbstständiges Lernen in kontrollierten Lernumgebungen zu ermöglichen.\ Dabei geht es um zahlreiche Fragestellungen, von denen hier exemplarisch nur einige wenige genannt werden sollen: * Wie kann die individuelle Rate des Vergessens oder Verlernens festgestellt und in individuelle Lernempfehlungen integriert werden. * Wie kann ein individueller Lernrhythmus so festgestellt werden, dass die daraus abgeleiteten Lernempfehlungen zur erfolgreichen Wissensaneignung führen. * Wie kann das Niveau der Lernkontrollen dem bisherigen individuellen Lernfortschritt so angepasst werden, dass es der Performanz der und des Lernenden entspricht und gleichzeitig das Erreichen der Lernziele sicherstellt. #### 1.2 AutorInnenaspekt Ein weiteres zentrales Merkmal unseres digitalen Lehrformats soll die **umfassende Unterstützung von AutorInnen bei der Erstellung digitaler Lerninhalte** sein. Gerade in der innerbetrieblichen Aus- und Weiterbildung übernehmen häufig Fachexpertinnen ohne pädagogische oder didaktische Ausbildung die Rolle von TutorInnen – und damit auch die Verantwortung für die Qualität der Wissensvermittlung. Dieser Herausforderung kann mit einer Vielzahl KI-gestützter Werkzeuge begegnet werden, die eine strukturierte, lernwirksame und mediengerechte Aufbereitung von Inhalten ermöglichen.\ In der akademischen Lehre werden Lerninhalte meist über Jahre hinweg von Personen mit didaktischer Expertise entwickelt und gepflegt. Diese Inhalte zielen auf übergreifende Bildungskompetenzen und sind in der Regel stabil strukturiert. *In der betrieblichen Realität hingegen sind die zu vermittelnden Wissensgebiete oft kleinteiliger, stark kontextbezogen und unterliegen einer hohen Veränderungsdynamik*. Inhalte müssen oft schneller, zielgruppen-spezifischer und adaptiver erstellt werden.\ Wir gehen davon aus, dass AutorInnen im betrieblichen Kontext Lehrthemen zunächst grob beschreiben und mit einer Sammlung vorhandener Materialien wie Texten, Bildern oder Videos anreichern. Auf dieser Basis generiert unser Digitalformat Vorschläge für eine geeignete inhaltliche Struktur, inklusive Einleitung, Kernabschnitten und Lernzielkontrollen. Die Inhalte werden in einem didaktisch sinnvollen Aufbau aufbereitet, abgestimmt auf Zielgruppe, Lernziel und Praxiskontext. Dabei greifen integrierte Assistenten auf bewährte Modelle der Erwachsenenbildung zurück und unterstützen z. B. die Formulierung von Lernzielen oder die Auswahl passender Präsentationsformen und -medien.\ Die konkrete Umsetzung mit einer automatisierten Aufbereitung unterstützt die Überführung von Inhalte aus Fachtexten, Präsentationen oder Videos in interaktive Lerneinheiten, inklusive automatisch generierter Lernkontrollen, Reflexionsimpulse oder anwendungsnaher Aufgabenformate. Darüber hinaus soll sie stilistische Hilfestellungen bieten, etwa zur gendergerechten Sprache, zur Verbesserung der Verständlichkeit oder zur sprachlichen Vereinheitlichung.\ Die Wiederverwendung von Content-Bausteinen, die einfache Pflege dynamischer Inhalte sowie die Versionierung für kollaboratives Arbeiten soll AutorInnen nicht nur bei der technischen Erstellung, sondern auch bei der didaktischen Konzeption zu entlasten, ohne dass sie selbst über tiefere medienpädagogische Kompetenzen verfügen müssen.\ Gleichzeitig ergeben sich aus dieser Form der KI-gestützten Unterstützung für AutorInnen spezifische Risiken, die im Entwicklungsprozess aktiv adressiert werden müssen. Dazu zählen insbesondere: * Unreflektierte Übernahme automatisierter Vorschläge, was zu inhaltlich oder didaktisch unangemessenen Lernangeboten führen kann. * Fehlende Kontextsensibilität: Formal korrekte Inhalte können im betrieblichen, interkulturellen, sprachlichen oder rechtlichen Kontext unpassend sein. * Didaktische Verflachung, wenn AutorInnen sich zu stark auf die automatischen Strukturen verlassen und dadurch kreative, zielgruppenspezifische Lösungen unterbleiben. * Mangelnde Konsistenz bei mehreren AutorInnen, insbesondere bei kollaborativem Authoring ohne redaktionelle Abstimmung. * Urheberrechtliche Risiken, wenn KI-generierte Inhalte mit externem Material angereichert werden, dessen Nutzungsrechte unklar sind. * Überforderung unerfahrener AutorInnen, wenn die Vielzahl an Optionen und automatischen Vorschlägen zu Unsicherheit oder Fehlentscheidungen führt. Unser Digitalformat wird diesen Herausforderungen durch eingebaute Plausibilitätsprüfungen, Feedbackmechanismen, kontextabhängige Warnhinweise und optionale Review-Funktionen begegnen. Ziel ist eine Plattform, die nicht nur Inhalte erstellt, sondern AutorInnen bei der pädagogischen Reflexion begleitet und weiterbildet. Die Kombination aus adaptiver Inhaltsgenerierung und didaktischer Assistenz eröffnet neue Wege für qualitativ hochwertige, betriebsnahe Lernangebote. Sie bringt aber auch Herausforderungen mit sich, für die es bislang kaum technische Standards oder erprobte Prozesse gibt.