On the use of historical data in context-aware multimedia documents adaptation processes

,


INTRODUCTION
Ambient intelligence aims to create environments with communication, computation and decisionmaking capacities.These features help to enhance the way in which users interact with various environments, based on real-time information gathered and historical data accumulated.Ambient intelligence is built around ubiquitous systems (known also as pervasive systems) by introducing a key-element: explicit requirement of intelligence.This way, mobile applications can sense their environment, understand the context of collected data and adapt their behaviour accordingly [1].The context refers to information used to depict the situation of entities considered relevant to the interaction between a user and its environment.
In this work, we consider the case of playing multimedia documents in ubiquitous systems.These documents include media objects of various natures such as text, image, audio, and video; web pages are a good example.This type of documents plays an important role in several fields such as healthcare, distance ISSN: 2302-9285  On the use of historical data in context-aware multimedia documents adaptation processes (Aziz Smaala) Generally, these data are obtained either by events triggered by context changes or by time driven methods according to which the context values are calculated during periodic checks.b.Thinking layer: this layer comprises three steps namely context-modeling, context-reasoning, and decisionmaking which act as follows: -Context-modeling: the context information collected is raw because it only includes values measured by various sources (low-level context data); hence, it should be analysed further to take desired actions.To this end, the gathered information is organized according to an abstract context model such as key-value models, markup scheme models, graphical models, object-oriented models, logic-based models, and ontology-based models [22] (high-level context data).The goal is to allow processing context information at a higher level that helps to reason upon it.-Context-reasoning: the reasoning step involves the high-level context data to identify the constraints and conflicts that hinder playing the documents properly; it usually takes place in two forms: real-time reasoning and on-demand reasoning.Real-time reasoning refers to event driven and time driven context; hence, it is continuously executed in order to infer the required adaptation actions.On-demand reasoning is triggered by a request from a user or application for many purposes such as resources discovery (e.g.surrounding users/objects) and recommendations (e.g.suggestions regarding environmental conditions such as lighting).In the end of this step, a set of adaptation actions are inferred in order to provide adapted documents that fit as much as possible the contextual situation, often through "if ... then" rules.-Decision-making: the objective of this step is twofold: the determination of the adaptation actions to be executed and the generation of the adaptation plan.The decision-making step deals with the adaptation actions produced in the context-reasoning step in three possible modes: i) the automatic mode according to which the system keeps these actions without involving the interaction with the user, ii) the semi-automatic mode according to which the system proposes to the user the inferred actions, in order to choose the most suitable among them, and iii) the manual mode according to which the adaptation actions are entirely generated by a request from the user based on specific needs.
Once the set of adaptation actions has been determined, the adaptation plan is generated by binding each action to a set of possible execution paths, depending on the available candidate adaptation services.Then, the best paths are selected based on a service selection procedure according to QoS parameters (e.g.response time).c.Acting layer: the last step consists in performing the adaption plan by executing the selected adaptation services, in order to meet the requirements of the environment and the user.These services are applied to media objects composing the documents to produce a set of adapted objects.These contents are then arranged in an adapted document so that it is displayed in a suitable form to the current contextual situation.
Let us consider a mobile user consulting online course.For the sake of simplicity, we assume that the context elements include the location, current activity and preferred language of the user.Table 1 gives some typical context values, conflicts, and adaptation actions related to the MDA process used.

Involving historical users data in context-aware multimedia documents adaptation processes
In the litterature, much research has been conducted on context-aware maultimeida adaptation [2]- [21].Depending on where decision-making and adaptation actions take place, MDA processes can be  [23].The adoption of a particular category depends on many factors such as devices computing features, the communication bandwidth, and the available resources.a. Client-side adaptation: the devices running the documents make the adaptation process by themselves (e.g.[17]).This option allows more responsiveness; however, the limited resources of some devices (e.g.battery, CPU) may be a hindrance to the execution of adaptation actions.b.Server-side adaptation: the devices playing the documents send adaptation requests to a server that deals with them (e.g.[20]).Although this option frees the client side from a tedious process that could slow down performance, this may affect the server response time due to overload.c.Proxy-based adaptation: a proxy is placed between the client and the server in order to act as a mediator while running the adaptation process (e.g.[7]).The proxy frees both client and server from intensive tasks by leveraging its computing power; however, this option may involve too much communication for negotiating which adaptation actions to execute.d.Peer-to-peer adaptation: the devices playing the documents can communicate with multiple platforms, including other devices, to perform the adaptation process (e.g.[5]).This option takes advantages of the peer-to-peer paradigm to ensure a balanced load while executing the adaptation actions.Despite this, the process efficacy depends on the number of connected nodes.Furthermore, the network administration is difficult due to its decentralized nature.
Table 2 provides a comparative summary of some of the MDA approaches cited above.The aim is to compare them according to their main limitations concerning the way the reasoning phase is performed.Such a comparison helps propose improvements and contributions with respect to related research-works.[5] Peer-to-peer A service-based approach for MDA in which adaptation services are available locally and remotely on multiple platforms.
No support of semantic aspects in profiles and adaptation services.Dromzée et al. (2013) [7] Proxy-based A service-based context modeling approach for MDA that links the context information using semantic generic profiles.
No support of reasoning mechanisms.Bettou and Boufaida (2017) [11] Server-side An MDA approach based on formal concepts related to technical adaptations to select relevant policies by detecting the conflicts between documents properties and users preferences.
No support of semantic aspects to deal with context constraints.

Adel et al.
(2017) [13] Server-side An MDA approach that involves social impact inferred from communities on Twitter and Facebook, to make an effective composition of adaptation services using semantic relationships.
No support of the compatibility between the inferred adaptation actions.Saighi et al. (2017) [12] Server-side A handicap-based MDA that exploits an ontology to provide context-aware assistance by binding each context constraint to a physical handicap and thus to an adaptation action.
Only one adaptation action is inferred per adaptation request (no support of multiple actions).Khallouki and Bahaj (2017) [14] Server-side A context-aware MDA approach based on cooperative multiagents systems that uses semantic web services to predict users context and provide adapted contents accordingly.
No prototype was implemented for the proposal to be tested and evaluated.El Guabassi et al. (2018) [16] Server-side An XML-based personalization of course content in ubiquitous learning, considering learning styles and context-awareness No support of semantic aspects to deal with context elements.

Belhadad et al. (2018) [17]
Client-side An XML-based multimedia specification that describes the structure of web contents according to users context and profile.The content is adapted by modifying their spatial structures.
Only the spatial dimension of objects is handled but not their temporal synchronization.Saighi et al. (2020) [20] Server-side A graph-based MDA that allows the inference of multiple adaptation actions using semantic reasoning upon users context.
No parallel computing support for reasoning (performances issues).
By analyzing several MDA approaches [2]- [21] (see Table 2), we could check that they focus mainly on context collection, representation, and interpretation.In particular, the reasoning mechanisms only rely on profiles and current context values in such a way that the adaptation task is performed in a single location (i.e.client, proxy or server sides).Thus, the storage and analysis of past context values and corresponding adaptation actions are not supported in order to be involved within adaptation processes.Logging of history information is supported by many context-aware pervasive applications as it helps to provide prior knowledge about users decisions so that they can be taken into account in future processing.We cite as examples machine-learning approaches for: IoT cultural data [24], context-aware intrusion detection [25], smartphone data analytics [26]- [29], activity recognition [30], and healthcare support [31].
Likewise, HUD can contribute in improving the overall behaviour of MDA processes by performing several advantageous tasks such as machine-learning techniques, statistical methods for building recommendation systems, and adaptation rules personalization.In this case, two types of HUD can be considered either jointly or separately.The first type concerns the multimedia documents consulted by emphasizing on general information about users and resources accessed.[32].In contrast, there is still a general dearth of MDA approaches treating the second type of HUD.Indeed, even though there are some MDA-based applications that involves certain HUD (e.g.[33], [34]), they however do not provide effective mechanisms to manage such data since they only focus on analyzing them either through datasets or by certain collected data.Generally speaking, log data are extremely diverse and processing them is unfortunately quite complex.This is due to the lack of standards and agreements outlining the granularity, structure, content, format, and level of details provided by log events [35].This raises certain issues related to their management, which are further addressed later in this paper: -The first concern is about the determination of which data will be stored; this is a crucial question that depends on the intended purposes of their storage.-The second concern is about the format in which the data will be stored.This aspect has a direct link with efficient data storage and retrieval for further processing.-The third concern is about the place where the data will be stored.This is a very important aspect since it may lead to dealing with large volumes of data (most likely big data).-The last concern is about the way in which the data stored will be leveraged in useful tasks by integrating users experiences into MDA processes, in order to improve their effectiveness.
Starting from this context, we propose to design a software component that handles HUD resulting from the execution of MDA processes for further processing.Compared to other MDA-based approaches, our proposal should be generic insofar as it can be integrated into different MDA processes categories in such a way that it allows the storage, retrieval and analysis of historical data as log data resulting from the accumulation of the decisions made by users.This way, the proposed component will provide MDA processes with more options to efficiently process HUD in different locations namely clients, proxy, and servers sides.To do so, we rely on a well-devised, flexible, and agile architecture that can easily change form (polymorphism), and adapt to context changes (e.g.computation resources), and users preferences/requirements (e.g.personalized processes, data privacy, and sharing).

METHOD
Our proposal consists of a software component for managing HUD resulting from the execution of MDA processes.The aim is to allow their storage, retrieval and analysis so as to involve them in users decision-making according to different ways.This is a complementary element to existing adaptation approaches that attempts to improve their overall performances since in most of them: a. Decision-making is performed by inferring the adaptation rules using only on-demand and/or real-time reasoning; thus, they do not imply HUD when performing the adaptation task.Our proposal allows MDA processes to leverage from HUD to perform several advantages tasks (e.g. machine learning tasks), in order to improve the adaptation process quality.b.The whole adaptation task is carried out in a single location (clients, proxy or server sides), in particular the client-server paradigm which is broadly adopted due to implementation simplicity and computation resources power.Even so, the efficient use of other paradigms can also be a suitable choice to deal with several difficulties (e.g.workload balancing).One way to take into account this issue is to rely on an effective hybridizations between different adaptation categories such that the MDA process is executed in a decentralized way by switching from one adaptation category to another in the event that the current solution shows its limitation.In this regard, our proposal is built around a well-devised, flexible and agile architecture that offers the possibility to treat HUD in different locations and in different ways, which: -makes the HUD manager potentially supported by all MDA categories.
-provides more options for MDA approaches, depending on the current computation resources and users preferences (e.g.data privacy and data sharing), and -allows HUD to be managed by different devices (e.g.smartphones, laptops, and servers).

The general architecture of the proposed HUD manager
Now, we give details of the proposed HUD manager.Figure 2 shows its architecture and the potential interactions with MDA processes.The proposal comprises four components as follows: a.The database: it represents the storage medium for storing the HUD.b.The data logger: it makes systematic recordings of HUD in the database each time new data is acquired by the system.c.The data retriever: it allows retrieving HUD from the database to be used by the data analyser.d.The data analyser: it mainly consists of a bag of data analysis algorithms dedicated to different purposes such as machine learning techniques, and data mining methods.The interaction between the MDA processes explained in subsection 2.1 and the HUD manager takes place through an invocation of the data logger and the data analyser, performed by the thinking layer, as shown in Figure 2.This layer includes the context-reasoning and the decision-making modules that constitute the core of the adaptation system and therefore the source of its intelligence.On the one hand, the decisionmaking module invokes the data logger to store HUD about the context values and the decisions made by users (step 1 in Figure 2).On the other hand, the context-reasoning module takes advantages of the functionalities provided by the data analyser that contribute in making future decisions (step 2 in Figure 2).Next, we summarize the three steps executed by the proposed HUD manager.
-Step 1: HUD logging Once the decision-making module has generated the adaptation plan, it calls the data logger component to store information about the corresponding context values and adaptation actions made by the application.In turn, the data logger accesses the DB through a query language-that depends on the implementation-in order to insert these data as new entries (see Figure 2).

-Step 2: data analysis
The data analyzer component provides a bag of data analysis algorithms that can be used by the context-reasoning module, in order to improve its effectiveness.To do so, it calls the data retriever to acquire data for analysis according to two methods: real-time analysis and on-demand analysis.Real-time analysis refers to event driven methods triggered whenever new entries are inserted into the DB; it is continuously executed in order to update the context-reasoning module as needed, depending on the purposes of the application.Algorithms in this category should be lightweight in order to maintain the responsiveness of the system with respect to application usage.A typical example of this category is the statistical analysis methods based on pattern frequencies for which the algorithm needs only to maintain the frequencies using a dynamic data structure.On-demand reasoning is triggered by a request from a user or application for many purposes such as machine-learning algorithms and recommendation systems.Algorithms in this category can be heavyweight.Moreover, they can be executed online or offline as they do not influence the responsiveness of the system with respect to application usage.

-Step 3: data retriever
The data retriever is designed according to the data access object (DAO) pattern that provides an abstraction between several database types as well as other persistence mechanisms and the other applications parts.This way, calls to the persistence layer goes through the DAO so as data operations are performed without revealing database details.Thus, the data retriever consists of a set of callable methods that retrieve data from the DB according to different input parameters.The implementation of such methods is based on query languages (e.g.SQL, and XPath) depending on the type of DB adopted.Two types of data retrieve methods are proposed: data retrieve based on triggers and data retrieve based on query languages.The first type of data retrieve uses triggered callable methods that are executed when certain conditions related to DB updates are met.The second type of data retrieve is based on parameterized callable methods that access and retrieve data from the DB according to the input parameters.
The proposed HUD manager has several functions.This provides various options that help in efficiently designing MDA processes according to users requirements and context specifications.Depending on where the HUD is stored and processed, three methods in addition to their hybridization are proposed: a. Client-side management: the client devices store, retrieve and analyse HUD by themselves.

ISSN: 2302-9285 
On the use of historical data in context-aware multimedia documents adaptation processes (Aziz Smaala) 1117 b.Server-side management: the HUD manager is installed in a server that stores, retrieves and analyses the data.The devices playing the documents (i.e.clients) send requests by invoking the data logger, and the data analyser as needed.c.Proxy-based management: the HUD manager is implemented in a proxy that is placed between the client and the server.d.Hybrid management: the components of the HUD manager namely the database, the data logger, the data retriever and the data analyser, are separately placed in the client device, proxy or server, depending on the current requirements of the MDA process.

Implementation of the proposed approach
To show the feasibility of our proposal, we develop a real prototype through which multimedia documents are adapted according to contextual situations while storing the HUD.In fact, from a functional point of view, most MDA processes work very well.However, when it comes to validating a given adaptation approach, the prototypes developed are often based on simulation to collect context elements; mainly for the elements related to connected objects (see for example [11], [12], [20]).Indeed, even if the simulation paradigm is very useful in many situations, the realization of a real prototype remains the most ideal, realistic and viable solution, in particular to deal with technical, and implementation issues.

Context senesing
Context values come from different hardware and software sources.Initially, they are collected as raw data since they only include measured values.These values serve as input data for the reasoning phase, the purpose of which is to build adaptation rules.Table 3 details the context elements used in our prototype.

Reasoning upon the context
The reasoning upon context values takes place in two phases: qualification of context values and inference of adaptation actions.The first phase aims to qualify the quantitative values of the context so that they are effectively used during adaptation actions inference and HUD analysis.Table 4 shows typical examples of context values quantification; these values are usually defined empirically.
Once the context values are qualified, the second phase is to infer the adaptation actions.Each adaptation action consists of a rule taking the form of "if (conflicts) then (actions)".Table 5 gives details of a subset of typical adaptation actions.To build these rules, we are inspired by those used in [12], [20].

Execution of the adaptation actions
Our prototype supports web contents adaptation according to user contexts without depending on a specific domain of application.The aim is to ensure the genericity of the prototype so that it can be adapted to a wide range of applications using multimedia documents with simple objects (e.g.texts, images, and videos) or those composed of a combination of different types of contents.Generally, there exist several types of document sharing, depending on applications scopes; we cite as examples file transfer protocol (FTP), peer-to-peer sharing and cloud service sharing (e.g.[36], [37]).Regarding our prototype, the HTTP protocol was adopted to share a set of multimedia documents hosted on a server, due to simplicity of implementation.
The adaptation of multimedia documents takes place by involving a set of adaptation actions.Each adaptation action is performed by executing one or several adaptation services, depending on the nature of the media objects in the documents.The description of the adaptation actions given in Table 5 is as follows: a. Speech to text: it mutes the sounds of auditory objects by replacing them with textual content (sounds on videos are muted and subtitled while audio contents are converted to text).c.Video to images: it converts videos to images sequences.d.Language translation: this action translates media objects written in a source language to the target language specified in the users profile.e. Format change: it changes the format of a media object to another format (e.g.mp3 to mp4).f.Video in smart TV: this action plays video objects in a smart TV since the screen size is larger.g.Text summarizing: this action summarizes the content of text objects.

Hardware and software configuration
The hardware configuration includes an arduino UNO board on which three sensors are connected: sound, GPS, and photoresistor light sensors.Our prototype uses WiFi and bluetooth low energy (BLE) connectivity.On the one hand, WiFi is one of the most popular wireless communication protocols.In addition, it is very compatible with a wide range of devices.On the other hand, BLE-based devices are very adopted in IoT implementations since they provide low power consumption and consistent connectivity.These protocols are used within the sensing layer shown in Figure 1, to collect the context elements from various sources.They are also used in the intercommunication between wired and wireless devices.Our prototype can easily be expanded so that it supports other communication protocols such as zigbee, in order to satisfy the potential demands for compatibility with installations based on other ubiquitous IoT protocols.
The hardware configuration also includes two HP Z640 stations (xeon CPU composed of 20 cores and 64 GB of RAM) used as a proxy and a server, and a smartphone (samsung galaxy A10s with helio P22 CPU and 2 GB of RAM) used as a client device on which the documents are run.Adaptation services calls are performed using the following java APIs: google cloud speech API (details can be found on the website: https://cloud.google.com/speech-to-text/),yandex translate (available at: https://tech.yandex.com/translate/),fast forward moving pictures expert group (FFMPEG) (downloadable at: https://www.ffmpeg.org/)and Jave (downloadable at: http://www.sauronsoftware.it/projects/jave/).
The database component is implemented using relational and NoSQL databases as they operate independently of any specific data analysis framework; the aim is to compare the two models.On the one hand, relational databases allow handling important volumes of structured data while offering efficient functionalities for data storage and retrieval.On the other hand, NoSQL databases allow handling large volumes of data at high speed with scalable architectures according to different degrees of data structuredness.Besides, there have been significant efforts made to build bridges between oriented-object, ontology and key-value paradigms on one side, and relational and NoSQL databases on the other side (e.g.[38], [39]).The data to be stored are restricted to qualitative values of the context elements and the corresponding adaptation actions resulting from changes in context over time.The static aspect of the HUD structure is specified by the class diagram shown in Figure 3.  6).Concerning the relational databases, we have used SQLite databases for client-side methods and MySQL databases for both proxy-based and server-side methods.Regarding NoSQL databases, we have used JSON files for all methods, in particular JSON files in hadoop distributed file system for sever-side approaches.
Regarding the implementation of the remaining components namely the data logger, retriever and analyser, we rely on the oriented-object paradigm features.More specifically, we use Java interfaces since they help to design processes specifications that can be implemented in different ways.Table 7 shows the implementation details according to the different data management methods.Java language for Android systems Proxy-based Java language for standard systems Server-side Java language for systems based on map/reduce paradigm

RESULTS AND DISCUSSION
To validate our proposal, three tests are conducted.The first test consists in a case study through which we show how the data are organized in the database as well as a typical example of the adaptation result of a multimedia document.The second test aims to measure and evaluate the performance of our proposal in terms of processing time and size of data required to process the HUD.Finally, the last test consists in a case study through which we show the usefulness of storing the HUD by considering a typical example of how HUD contribute to the personalization of adaptation rules.

Formating and storing the HUD
This section shows how HUD are formatted and stored in the database.Intuitively, there are two methods to transform the HUD modeled in Figure 3 into relational model and JSON objects.In the first case, the log data take a columnar form by grouping together all attributes of HUD classes; thus, the data are structured as a set of entries.This format is useful for applying data analysis techniques.In the second case, the log data take the form of structured entities (linked tables, and nested objects) using mapping procedures while maintaining the relationships among them (e.g.[38], [39]).This format is more suitable for data readability at a high specification level.Regarding our work, we adopt the former case since our main goal is to perform data analysis.Below, an illustrative scenario for which the HUD are stored by executing an adaptation process triggered under certain assumptions.For this purpose, we rely on the prototype developed by running a client-side adaptation system that deals with web pages contents according to the context situations of users while handling their HUD.We note that the approach is general so that it can easily be adapted to several dedicated MDA processes such as medical applications, ubiquitous learning, and tourism filed.Thus, the scenarios considered only represent demonstrations by way of non-limiting examples.
Mr. John is a medical-researcher at university; he generally connects to medical forums for discussion.Given the nature of his job, John is often on the move for consultations, and conferences.When traveling, he wants to consult a multimedia document related to radiology.The document is written in french; it consists of text, audio, image, and video contents, as illustrated in Figure 4. Let assume that John is in a conference room and uses his tablet to access the document.The battery level of his tablet indicates that it has reached 25%.The contextual constraints inferred by the adaptation system are as follows: public place, low battery and inappropriate language.Hence, rules Ri=1, 3, 4 in Table 5 are executed to provide an adapted document, as shown in Figure 5. On another side, the system invokes the data logger to store the log data in the database, according to the methods shown in section 3. A typical HUD input content is encoded as in the following JSON code.

Performance analysis 4.2.1. Running time and data size measurements
Next, we evaluate the different database models shown in Table 6.The evaluation of performance is carried out by measuring the running time and the data size required for processing the HUD according to the current number of entries in the database.In particular, we study the average time for inserting one HUD entry in the database, the required time for loading the database content in memory to make data analysis and the storage space required for storing the HUD.The tests are performed using the hardware configuration mentioned in subsection 3.2.4. Figure 6 shows the obtained results by assuming that each user stores a maximum number of entries=10000.The users number is set to 1 for client-side HUD management and to 50 for proxy-based/server-side HUD management.Figure 6 includes: -Figures 6(a    .This is expected as the volume of processed data becomes larger.Nevertheless, relational databases are well performing for data retrieve compared to NoSQL ones since the latter parse data before storing them as objects in memory.-The required space for storing data increases linearly in accordance with the growth of databases sizes (see Figures 6(e) and (f)).Besides, relational databases require less storage space given that NoSQL databases store entries as combinations of keys (i.e.objects and attributes), and values.To overcome this issue, it is possible to make the descriptors length as shorter as possible.Even so, NoSQL databases may excel since they do not require dedicated configurations either for exporting data or for accessing them in addition to their simplicity of implementation.Globally, the general performances of the proposed HUD management methods depend mainly on the available computational resources.For instance, during the experiments, we could check that the clientside method could not store more than 55.000 in SQLite databases.It also could not load more than 150.000 entry in memory from NoSQL databases.Likewise, the proxy-based method could not load more than 800.000 entry in memory.This leads us to care about issues related to efficiency of data storage, retrieval, analysis, scalability and others.

Comparison of the proposed HUD management methods
Now, we compare the proposed HUD management methods detailed in subsection 3.1 with each other based on systems ability to meet the following criteria: data volume, data performance, data retrieval, data analysis, scalability, data sharing, and data security.These criteria depend on certain factors that influence the general performances of the HUD management system such as data storage location, data size, computation resources, and number of users.Table 8 provides details about each criterion used for comparison purpose.

Less suitable Unsuitable
Data security Ability to deal with privacy and security of users data.

HUD storage location
Strong Weak Very weak Table 9 summarizes the comparison between the proposed HUD management methods according to the criteria presented in Table 8.The statistics given in Table 9 help to determine the way in which the HUD can be effectively managed, and processed.Globally, server-side methods may excel because of the computation resources they have, the number of clients they can handle and the data volume they can store.In contrast, client-side methods show high data security since they store them on clients devices; thus, any access to data goes through client permission.
Even so, these features may be affected by several factors such as the available computation resources, the current workload, load balancing issues, fault tolerance, and service availability.In such cases, hybrid management methods (i.e.those combining client-side, server-side, proxy-based, and peer-to-peer paradigms) help overcome these issues through an efficient placement of the HUD manager components in different locations, depending on the needs and requirements of both applications and users.

Comparison of our proposal over other approaches exploiting HUD
Finally, we compare our proposal with three research-works [33], [34], [40] that exploits HUD in pervasive environments (see Table 10).The purpose is to show the strengths of our proposed approach as well as the value added so as to improve MDA processes.To do so, we first consider the following points: -Approach scope: it describes the scope of a given research-work.
-Approach category: it refers to the HUD management methods discussed in section 3, namely server-side, proxy-based, and client-side.-Efficient historical data management: it refers to whether a given approach uses/provides efficient mechanisms to handle the HUD.-Technical details: it refers to whether a given research-work provides technical details about HUD handling.-Historical data use: it refers to the purpose for which the HUD is leveraged in useful tasks.
-Historical data involved: it refers to the elements composing the HUD.
-Approach flexibility: it refers to whether a given approach is agile and generic so that it can be adapted to a wide range of approach categories.By relying on hybrid management methods (i.e.combining client-side, server-side, proxy-based, and peer-to-peer paradigms) our proposal may provide better data handling because several limitations of client-side, proxy-based, and server-side methods can be reduced (e.g.workload balancing, service availability) in particular if cloud computing techniques are involved.Thus, by considering the criteria given in Table 8, the potential strengths of our approach compared to the works presented in [33], [34], and [40] are summarized in Table 11.It should be noted that these values are assigned based on the details provided in the corresponding references.Moreover, the difficulty of reconstructing these context-awre systems and the lack of several technical details have prevented us from making numerical comparisons with our proposal.In view of the foregoing, we assess our proposal over related-works.The goal is to show the added values so as to improve MDA processes.The main advantages of our approach are summarized as follows: -The proposal can be considered as a complementary element to existing MDA approaches since most of them do not efficiently handle the HUD.-The proposal shows adaptability insofar as it handles the HUD at a high level of abstraction, regardless of any specific MDA approach (e.g.medical applications, and ubiquitous learning).-The high flexibility as the proposal offers various options to store, retrieve and analyze the HUD.These features allow overcoming some of the issues discussed in Table 9 through an efficient placement of the HUD manager components in accordance with applications and users needs.

Case study on HUD analysis: context-aware adaptation rules learning
In what follows, we consider a case study to show the usefulness of storing and processing the HUD.Thus, the scenarios below give a typical example of how HUD contribute to the personalization of adaptation rules through rule learning mechanisms.This task refers to prediction of adaptation actions using data-driven approaches (i.e. based on data analysis and interpretation) rather than explicitly defining them.
-Scenario 1: David is a mobile user.While moving, he accesses multimedia documents on various subjects through the mobile data accessible from his smartphone; this is a paid and limited service.To save mobile data consumption, he always asks the system to adapt the documents by using low quality media objects format.-Scenario 2: Sarah is a university student.Back home, she has to revise her online courses using her laptop.At that time, Sarah always feels tired due to her busy daily agenda.Therefore, she asks the system to adapt the target documents by summarizing their contents.These scenarios are implemented in a way that HUD are stored in a NoSQL database, denoted by DB, hosted in the proxy.For each scenario, we store two types of HUD.The first type concerns the storage of a sufficient number of entries according to the assumptions mentioned in the scenarios.The second type consists in storing random entries to endow database DB with dynamicity (i.e.noise) when analysing data.In order to infer new personalized adaptation rules, the process given in Figure 7 is proposed.First, the data analyser invokes the data retriever to load database DB as an array of objects, denoted by O.Then, array O is split into subparts Oi= 1 ... n according to user id; n is the total number of users.Finally, the elements of each subpart Oi=1 ... n, denoted by Ii={oij}, are analysed to extract associated conflicts and adaptation actions.-Let A={a1, a2, …, ak} be a set such that each element refers to one of the adaptation actions; k is their number.Each element aj=1..k ∈ A can be either 0 or 1 to specify whether adaptation action aj is considered for execution or not.-The backtracking algorithm goes through paths built by combining the possible values for vector X and adaptation actions from set A. When the confidence degree of any given combination (x, a) exceeds a threshold T, it will be considered as a personalized rule for the i th user.The confidence degree is defined as the ratio of (the number of objects from Ii={oij} including non-null values from vector x and executed actions from vector a) to (the number of objects from Ii={oij} including all non-null values from vector x).
The resulting rules take the form "IF (⋀ ci ∈ x / ci ≠ null) then vj=1..k ∈ a / vj = 1".In addition, each time a potential rule is retained, a redundancy check is applied in order to decide whether or not to keep this rule.Table 12 shows typical rules retained during the analysis phase.The above scenarios show the importance of processing the HUD to build intelligent context-aware applications through adaptation rules learning to predict users behaviours.Thus, the main advantages of such a task are summarized as follows: -The use of unsupervised machine learning mechanisms; therefore, the context-aware adaptation rules are learnt from HUD instead of defining them in a fixed way.Moreover, there will be no need to pre-trained data (datasets) to perform the learning task.-Possibility to limit the log data so that only some subparts are processed according to a sliding window (e.g.considering data corresponding to recent periods).-Reusability since it can be adapted to other rule-based context-aware pervasive environments.In spite of these advantages, there are still some limitations that should be overcome in future: -The system does not involve efficient association rules algorithms (e.g.frequent pattern growth algorithm).Indeed, backtracking algorithms cannot be applied to process large volumes of data.That is we need to develop more efficient and effective algorithms for data analysis in order to extract personalized adaptation rules.-As the system is based on unweighted (blind) correlations, it may take a long time to overlook the most frequent old rules and replace them with the most recent ones.-The system lacks of experience sharing between users; this feature helps build rule-based recommendation systems.

CONCLUSION
The aim of this work has been to involve relational and NoSQL databases to efficiently manage HUD resulting from the execution of MDA processes in context-aware pervasive systems.To achieve this goal, the context as well as the adaptation actions were first modeled through the oriented-object approach and then converted into appropriate schemes.To make the proposal adaptable to several adaptation approaches, four management methods have been proposed: client-side management, proxy-based management, and server-side management and hybrid management.The proposal has been validated through scenarios implemented in a real prototype with a special focus on use cases about context-aware adaptation rules learning.In fact, as the aim here was to show only the feasibility of the approach, the developed prototype was built around a restricted subset of general adaptation rules independent of any specific field of application.For real-world applications, the adaptation rules base may be broadened by considering other context elements, depending on the needs of the field of application.In addition, more sophisticated technical choices can be involved, such as sensing as service paradigm.Currently, we are working on applying machine-learning techniques along with other efficient data analysis algorithms, in order to perform contextaware adaptation rules learning from HUD.

Figure 1 .
Figure 1.The general architecture of a standard MDA process

Figure 2 .
Figure 2. Detailed architecture of the proposed component and its potential interactions with MDA processes

Figure 3 .
Figure 3.The static aspect of the abstract structure of the HUD

Figure 4 .
Figure 4. Structure of the original document

Figure 5 .
Figure 5. Structure of the resulting adapted document ) and (b) to plot the variations of the average time required for inserting a new entry in databases.-Figures6(c ) and (d) to plot the variations of the average time required for retrieving data entries from databases.-Finally,Figures6(e ) and (f) to plot the variations of the size of the data stored in databases.By analyzing the curves illustrated in Figure6, one observes that for both cases (i.e.client-side, and proxy-based/server-side HUD management): -The time required for storing one entry is quasi-constant regardless of databases sizes (see Figures6(a) and (b)).In particular, NoSQL databases are well performing for data storage compared to relational databases.This is because relational database management systems use advanced techniques to optimize the data storage, which affects the processing time.Bulletin of Electr Eng & Inf ISSN: 2302-9285  On the use of historical data in context-aware multimedia documents adaptation processes (Aziz Smaala)

Figure 6 .
Figure 6.Statistics about running time and data size required for processing HUD: (a) time for inserting one entry in client-side methods, (b) time for inserting one entry in server/proxy methods, (c) time for retrieving data in client-side methods, (d) time for retrieving data in server/proxy methods, (e) data size in client-side methods, and (f) data size in server/proxy methods 1123

Figure 7 .
Figure 7. Process phases for adaptation rules learning from HUD Bulletin of Electr Eng & Inf ISSN: 2302-9285  On the use of historical data in context-aware multimedia documents adaptation processes (Aziz Smaala) 1125

Table
. Some typical context elements, conflicts and adaptation actions

Table 2 .
A comparative summary of some MDA approaches The second type is related to

Table 3 .
Different context classes features

Table 4 .
Qualification of quantitative values of typical context elements

Table 5 .
Some typical rules for carrying out adaptation actions

Table 6 .
Technical choices to implement the database component in the HUD manager

Table 7 .
Technical choices to implement the remaining components in the HUD manager

Table 8 .
Criteria for comparing the proposed HUD management methods

Table 9 .
Comparison between the proposed HUD management methods On the use of historical data in context-aware multimedia documents adaptation processes (Aziz Smaala)

Table 10 .
Characteristics of our proposal and some context-aware approaches involving HUD

Table 11 .
Comparison of our proposal against some context-aware approaches involving HUD

Table 12 .
Typical rules retained during the analysis phase