Category Archives: projects

My LASI-NL 2014 experience

I was participating in LASI-NL event on 30th June and 01st July where I identified myself as the representative of the EMMA project and addition to LASI-NL event I had a chance to discuss several EMMA issues with our OUNL partners and plan the deliverable and share some EMMA learning analytics insights.

But back to LASI-NL event, where I was one of the fewest foreigners.  The first day was opened by the Dr. Stefan Mol and Dr. Gábor Kismihók from the University of Amsterdam. They shared their experiences with the learning analytics, faced barriers and mostly – how to overcome the barriers. They discussed the aspects of using learning record store (LRS) and Tincan API/xAPI specification. They don’t use existing LRS, as I understood, but aim to develop own LRS what stores data from different data sources.

From practical field, the presenters described their practical experiences with LA. The first cluster, as they named it, focused on mirroring of traditional and non-traditional study performance to students. Their tool (UvA Mirror (Coach)) – visualizes the position of individuals in the context of the group. Another tool qDNA (cluster exam feedback) provides fine grained mirroring of exam results to provide students and teachers with insight in the development on four competencies and knowledge goals. Provided dashboard facilitates individual students to choose from, and set their own goals (and deadlines) against specific course events deadlines based on mirrored data. Dashboard shows them in a glance how they are scoring/succeeding in attaining the goals compared to their fellows.

Second cluster focuses on providing feedback to teachers based on specific student data.  Tool visualizes scores for students and teachers over (partial) assignments or courses, provides web lecture statistics to find potential problems, difficult material to improve teaching etc.

The third cluster uses other people’s data to provide recommendation system to students and aims to reduce dropouts in bachelor programs by first find predictors and then start developing interventions.

The list of lessons learnt identified by the presenters, was quite long. One of the interesting remarks was that data is everywhere, but accessibility is the issue and additional legal and ethical issues are on the agenda. I also noticed that in Netherlands the LA is in national level important direction and funds are available for the R&D activities on the field.


Second session was about stakeholders’ requirements and Hendrik Draschler and Maren Scheffel from Lace project introduced their research. Since 2012 they have had three studies that involves LA stakeholders. First study in 2012, with the 121 participants (75% from the field of higher education) illustrated that 70% of the participants are quite sure that students are not able to learn from the learning analytics.

Two another studies were mainly interesting for me because of their data collection and analysis methods – group concept mapping and point mapping seemed like methods that I’d like to try. Results of the second study they conducted in OuNL indicated that Netherlands is ready to roll out LA in K12 and higher education. Third study involved LA experts (55 all around the world) and based on their expertise and opinion the framework of LA quality indicators was developed. LA experts considered the LA important as it provides the learning and outcome support and raise the students’ awareness of learning.

That session was interesting for me because I wanted to see what is the OunL LA research group and LACE project doing.


Next one was practical session. Session was guided by Dr Jeroen Donkers from WatchMe project. The WATCHME project aims to improve workplace-based feedback and assessment and professional development by means of learning analytics. Surprisingly the WP of the LA is coordinated by Tartu University and my opponent – small and exciting world.

After short introduction to the project and theoretical overview of how the LA and instructional design could complete each other, we were grouped for small groups and got the practical task: MOOC about educational management is being conducted for the second time. Drop-out of the course is high, small groups do not work optimal way and participants complain that content does not fit to their actual situation. The situation was best possible scenario for me, because it is directly associated with the work that I’m doing in EMMA. Not that surprisingly all the six groups has the same understanding what could be the solution. In our group we tried to take some instructional design model as a baseline (e.g. Kemp et al), but soon we came to conclusion that LA is very cyclical and the evaluation is part of the implementation and it is not enough to evaluate the iteration in the end. Also it was discussed that it is not important to focus on drop-out rate, but to direct those participant to collaborate who would like to collaborate. Anyway all groups seemed to commonly agree that it is important to: a) find out in the early stages the background of the participants in order to evaluate what is the interest in course and background (pre-screening of the participants); b) monitor the progress and redesign the course design including recommending the suitable materials based on the history and background, relevant study peers and study groups; c) post-evaluate the course.


From the second day I’d like to highlight the second keynote – Dr Ulriche Hoppe (University Essen-Duisburg, Germany). Title of his talk was intriguing to me –  Beyond the obvious – advanced analytics tools and applications in educational contexts. Simply said – “obvious” LA in his talk was the simple LA that focuses on frequencies of the activities, participation, enrollment. To be honest – something that is in my mind in EMMA as well. Non-obvious is analysis of sequences and paths that users are stepping on. To be honest – it is also in my mind, but is more complicated. Hoppe mentioned three LA methodologies – activity analysis, artefact analysis and network analysis and brought two of the non-obvious examples of his research, which were really close to artefact-actor-network analysis method that I’m trying to implement in EMMA as well. To be honest, some of his talk was far complicated to me (e.g. practical examples how to analyse), but I warned him that I’ll try his tools and contact him when needed. From Dr Hoppe talk I learnt that it is not important to visualize everything to the users –complex SNAs or patterns is relevant for the researchers.


Next session was too short for me, I wanted more. I’ve become familiar with the xAPI and with the concept of learning records store, but I wanted to hear about more and more. Anyway Alan Berg, Hendrik Draschler and Sander Latour shared their understanding and experiences with the xAPI and LRS. In theoretical level it seems that I’ve understood so far everything correctly and I know quite much. Sander’s practical experience was interesting because they have done something that is relevant for our Estonian LA project (eDidaktikum) – dashboard that provides overview of activities, comparison with others and recommendations by using xAPI standard. Activities in two different environments were combined, which is also interesting.

From Alan’s talk, I took with me two concepts – LTI protocol that allows to embed application to existing LMS and which works well with the xAPI standard; Caliper – framework and its relation with the xAPI.


To be concluded – it was extremely valuable experience for me. I positioned myself where am I and what are others doing. I understood how I’d like to proceed with the LA in my research group. I had a chance to meet my OuNL colleagues and make plans for EMMA. It has been long time since I was so excited 🙂


Course about technology-enhanced learning at workplace – reflection

This term had I new problem-based course with the IT-management master students about technology-enhanced learning at the workplace. Students in that course work mainly in public institutions – ministries as a IT managers.

The initial idea of the course was to  to support the enhancement of the knowledge and skills of the students to plan, implement and evaluate the technology-enhanced informal learning in the organization. Technologies that would support the workplace learning were planned to be technologies developed in Learning Layers.

Following phases were planned for the course: a) Students conduct the survey for finding out what kind of technology-enhanced learning practices can be identified in their organization, what could be changed and what kind of challenges can be faced with the focus on implementing technology for supporting learning; b) Students  design the technology-based learning activities and possible scenarios with the technological prototypes in the organization; c) Students evaluate the technology-enhanced learning scenarios with the technological prototypes.

The reality was a bit different, mainly of two reasons: the planned technologies were not in the state as initially planned and secondly, the institutions, where students are coming from, are quite restricted, so planning any new technologies is like.. impossible. I was encouraging them to imagine and dream that if it could be possible, how would you do then..

Most of them chose the technologies that the organization is already using, but not purposely. Most of the employees in their organizations are knowledge consumers who read newsletters, follow intranet or e-mails and never contribute to the organizational level knowledge. So they designed their scenarios with the focus on knowledge sharing and documenting the professional practices. As a result it turned out to be really interesting. Some of the evaluations of the scenarios with their colleagues turned out to be really successful, because they took the scenarios into real plan for the near future.

In the end of the course we had short reflection about the course. The main thing what they said was that next iteration of the course should not be about designing and evaluating so much, because it is more “managers’ issue” and less “IT-issue”. But they are more “IT-persons”. They would like to hear more about concrete technologies that are used in different SMEs, larger organizations for supporting learning, knowledge sharing. Based on the different research results, I could introduce the learning and knowledge sharing practices that work in different organizations. So, lets see.

For myself the course was extremely pleasant experience. I’ve never had such a group of learners and I’m glad I had a chance to teach them and to learn from them.

Update – I wrote short description of the course to Learning Layers Open Design Library as well, can be found here.


New project – EMMA started

I just arrived from Naples, kick-off meeting of the EMMA project. We, in the Centre for Educational Technology in Tallinn University, started with the new project – EMMA – European Multiple MOOC Aggregator, funded by 7th framework program, European Commission. Project period is 02.2014 – 08.2014.

Project is coordinated by Universita Degli Studi di Napoli Federico II from Italy.  Consortium includes partners from Spain (Atos Spain SA, Fundacio per a la Universitat Oberda de Catalunya, Universitat Politecnica de Valencia), Italy (Ipsos SRL, CSP – Innovazione Nelle ICT S.C.A.R.L), Netherlands (Open Universitet Nederland), Portugal (Universidade Aberta), United Kingdom (University of Leicester), France (Universite fe Bourgone), Belgium (ATIT Bvba).

Project aims to pilot the EMMA – MOOC’s aggregator – for conducting the MOOCs in different European countries. Emma aims to showcase excellence in innovative teaching methodologies and learning approaches through the large-scale piloting of MOOCs, providing a system for the delivery of free, open, online courses from diverse European universities with the goal of preserving Europe’s rich cultural, educational and linguistic heritage and promoting cross-cultural and multi-lingual learning. MOOCs are a recent development, rapidly becoming a significant trend in higher education. The open nature of MOOCs provides opportunities for expanding access to higher education and creating a space for experimentation with online teaching and learning. This exploration of new approaches for higher education provision has generated significant interest from governments, institutions and others. Building on existing and proven technology, EMMA will provide an opportunity, even for smaller institutions, to share high-quality content. EMMA will provide a framework and infrastructure, while participating institutions will remain autonomous in their choice of design, methodology and tools. EMMA will operate in two main modes; as an aggregator and hosting system of courses produced by European universities; and as a system that enables learners to construct their own courses using units from MOOCs as building blocks. The EMMA Consortium aims at a multi-lingual, multi-cultural approach to learning by offering inbuilt translation and transcription services for courses hosted on the platform. The pilot will operate in 2 steps, first by making available a significant number of existing courses from project partners and/or MOOCs providers and then by bringing on board a second tier of universities keen to experiment with MOOCs. Pilots will run in 7 countries with a total of 16 MOOCs and will involve at least 60.000 users. Courses will be offered in the language of each country, in English, and the pilot will trial an embryo form of multi-lingual translation by offering courses in Italian and Spanish as well. Advances in learning analytics will feature in the analysis and evaluation work and a series of innovative approaches will be trialed to make the piloted service sustainable in the medium to long term.

In the project, we will:

– Develop the learning analytics methodology with the technical suggestions how to integrate EMMA and different approaches to analytics;

– Develop and conduct at least one MOOC

Project seems interesting and the learning analytics approach is really exciting for us. Certainly gives valuable insights for our other projects as well that develop with the learning analytics.

Kick-off meeting was nice and I can’t be happier that we have consortium that consists mainly South-European countries – what a city, weather, food and people we met.