Optimal resource allocation in networked control systems using viterbi algorithm

Received Jul 22, 2020 Revised Mar 3, 2021 Accepted Apr 7, 2021 This paper presents an optimal bandwidth allocation method for a networked control system (NCS) which includes time-driven sensor, event-driven controller and random channels. A hidden markov model (HMM) with a discretized state space is formulated for the random traffic to predict the network states using a suitable data window. Network bandwidth is allocated based on the predicted traffic state subject to bounds on the deterministic traffic that guarantee acceptable NCS performance and do not exceed hardware limitations. Bandwidth allocation uses l1 minimization of unmet bandwidth demand. A stability condition is derived for a variable but bounded sampling period interval. Computer simulation results show the effect of varying the number of discrete states for the HMM and the window width on bandwidth allocation. The results compare favorably with a published approach based on fuzzy logic.


INTRODUCTION
Maximizing data flow is critical in networked control systems (NCS) with minimal resource capacity where several control loops share a common network [1]- [3]. Limited network capacity can result in packet drop or delay. To eliminate delay or packet drop, which may destabilize or adversely affect the response of the NCS, network resources must be used efficiently [4], [5]. Consequently, optimization of resource allocation is a major issue in NCS [6], [7].
Effective optimization and resource allocation require a good mathematical model of the NCS. In a recent paper [8], Ge et al. investigated a Markov model and stochastic control strategy for NCS dynamics. Hidden markov model (HMM) of NCS was first proposed in [9]. The authors proposed a stochastic optimal controller design for an NCS with network-induced delay governed by an underlying Markov chain of unknown probability distribution. Using the Baum-Welch algorithm, they obtained the initial distribution and state-transition probability matrix. They assumed three states for the NCS network load: low, medium, and high. Using a sequence of observations, they identified the Markov model and used it to design a stochastic optimal state feedback controller for the NCS. Nilsson also modeled network states as low, medium and high based on network load with transition between states governed by a Markov Chain [10]. The states were not directly observable and an HMM model was used for the network with the NCS treated as a jump linear system. Based on the HMM, he designed an LQG optimal controller to assure system stability. W. Lim, et al. [11], the authors proposed a dynamic bandwidth allocation strategy for orthogonal frequency division 1525 multiple access passive optical networks (OFDMA-PONs). Given the dynamic and stochastic nature of network, they used an HMM to predict traffic states. They defined the traffic states with two variables, the mean and the contrast of the bandwidth request observations. The latter characterizes the degree of traffic variation. Dainotti et al. [12] also classified packet-level traffic based on HMM. They used several types of traffic data and estimates of packet size and inter-packet time to evaluate their approach.
Li et al. [13] used fuzzy logic to construct a bandwidth manager. They determined the lower and upper bounds for the assignable bandwidth using linear matrix inequalities and the resource constraints. Fuzzy logic provides a flexible feedback mechanism to manage NCS that accounts for an upper bound and a lower bound on the sampling period. The proposed approach saves bandwidth and improves overall control performance as compared to fixed bandwidth allocation.
Although a rich literature on bandwidth allocation exist, to our knowledge the problem of bandwidth allocation for a network shared by NCS and random network traffic was only addressed in an earlier paper by [14]. G. Cetin, et al. [14], prediction of future traffic used a conditional poisson model. Although this provided a basis for bandwidth allocation, the results can be significantly improved by improving the quality of traffic prediction.
This paper builds on our earlier work to provide better bandwidth allocation for a network with random channels. Multiple links including random traffic, time-driven traffic, and event-driven traffic sharing a bandwidth is assumed. Network simulation data is used to formulate an HMM model for the network. Unlike earlier work, the model has a large number of discrete states that provide a more accurate network representation. Using the current traffic of the random network, the traffic at the next sampling point is predicted utilizing the HMM. Bandwidth of the network is allocated depending on the resource demands of each individual channel by solving a convex 1 optimization problem based on the predicted traffic for the random channel and the known deterministic traffic. Although it is assumed that network capacity is timeinvariant over the planning horizon of the control system, if the capacity is known a priori, the proposed method is valid if the capacity changes with time.
With limited network capacity, resource demand can exceed capacity. This may cause packet dropout and/or delay. As in [14], 1 minimization with time-varying sampling period for the time-driven channel is used to allocate bandwidth for the random and event-driven channels optimally. If insufficient bandwidth is allocated for the time driven sensor channel. The sampling period for the time-driven channel is increased to reduce traffic. However, the upper bound on the sampling period should be determined to achieve sufficient stability and performance in the closed-loop NCS system [15], [16]. We derive a condition for the stability of a linear time-invariant plant with a bounded sampling period using the results of [17]. The main contributions of the paper are; a. The paper proposes bandwidth allocation method for a network that serves random and deterministic traffic. b. The paper provides a new adaptive HMM network model with multiple states that can be used for traffic prediction. c. The HMM model is based on a sliding data window to allow for time-varying traffic conditions. d. The paper presents a new stability condition for a linear time-invariant system with time-varying sampling period. The remainder of the paper is organized as follows: section 2 describes the HMM model for random data traffic to predict states using Viterbi algorithm. In section 3, the convex 1 optimization problem and the dynamic sampling problem are described. Section 4 provides simulation results and discussion. Section 5 is the conclusion.

HMM OF NETWORK TRAFFIC
HMM is a statistical Markov models whose states are not directly observable. Instead, an output that is stochastically dependent on the hidden states is directly observable. The observed outputs are related to the HMM states by observation emission probabilities. Transition between states is random and is governed by state transition probabilities which indicate the probability of going one state to another. A states sequence from an initial state to end state can be generated by switching from one state to another based on the transition probabilities. The observable sequence of outputs is related to the state sequence by the emission probability distribution [18].
To obtain a HMM of a network, random network traffic level is allocated to five possible classes, which are "very low(VL)", "low(L)", "medium(M)", "high(H)", "very high(VH)". These classes can be determined based on total network capacity and network characteristics. With no loss of generality, the classes are identified with integers and are expressed as = { , , , , } = {1, 2, 3, 4, 5}. Assuming that a NCS shares a common network with nodes that generate random traffic, let (k) pac/sec be the traffic of the ℎ random channel at time k, = 1,2, … , . The traffic conditions are defined as follows; Channel conditions for i th random nodes: where is the lower bound for class c and is the upper bound for class for the ℎ random channel. For five different classes and random channels, we need 5 HMM states. For example, consider the case of 3 random channels, i.e. = 3, we use the notation } and so on. The state-transition matrix, whose entries are transition probabilities from one state to the next, can be determined by using relative frequency if enough data is collected. The emission matrix determines the probability of network congestion based on HMM states given measured outputs.
The emission matrix is classified based on total random traffic as Let the random traffic be given by where is the total random traffic, and , = 1, … , 6, are traffic bounds for the five network classes.
Based on this classification, we can define the emission matrix to include the probability of congestion states according to the HMM states. Since we have 125 states and 5 observation states, the size of the E will be 5 × 125.
The HMM can be defined as = ( , , , , ) where is the state-transition matrix, E is the emission matrix, is the number of the states of HMM (traffic conditions), is the number of observation states(congestion conditions) and n is the window size [9]. The HMM model is shown in Figure 1 To find for window size n, we first determine the traffic classes from the available data using (1). Then we find the transitions probabilities using the relative frequency of state transitions. To find the emission matrix for window size , we use the ranges of (3).
Since network dynamics change over time, we determine the adaptive transition matrix and the emission probability matrix based on random network traffic data over a specified time window. Using observation 1 , 2 … . , of network traffic conditions data ending at the current time , we find the adaptive transition matrix and the emission probability matrix for window size n. The Viterbi algorithm [18][19][20][21][22] is used to predict the most likely states at time = 1, . . , + 1, we save the predicted state at = + 1. Then we shift the window by one step, and update the adaptive transition and the emission probability matrices. The Viterbi algorithm predicts the most likely state at time = + 1, . . . , 2 + 1. We save the predicted state at = 2 + 1 and the process is repeated to predict future states.
The Viterbi algorithm works as follows. Given the HMM model and a sequence of observations, we to estimate the most likely sequence of states [18]. The most likely sequence ending with = is obtained using the utility function The paths that lead to each state from the possible prior states are explored to find the path that has the highest probability. For each possible new state, only the highest probability path is saved using ′ +1 .
′ +1 = + log ( +1 | ( ′ | )) + log ( +1 | +1 ( +1 | ′)) where ′ is posterior state at time + 1, and +1 is the maximum cumulative log-probability achieved for ( + 1) ℎ time step from ℎ time step. Using (5) recursively, we determine the most likely paths from = 1 to = + 1 for window size and save the most likely transitions for backtracking iteration. The final posterior probability of the all sequence of states can be maximized by recursively maximizing the joint probability for each possible new state. The best final state at time = + 1 for window size n can is calculated as Later backtracking iteration is used to find all hidden states in the window (traffic conditions) Figure 2 shows general structure of NCS. Random nodes occupy bandwith of the network based on its network traffic, sensor, controller and actuator nodes are interconnected via network for data transmission as in shown in of , we can assign random traffic for each random channel at time + 1 by assigning the upper bound related to the traffic condition to guarantee the required bandwidth allocation for the time-driven sensor node.

BANDWIDTH ALLOCATION
Since the total network bandwidth capacity is constrained, bandwidth is allocated optimally depending on the traffic of each channels. The Viterbi algorithm predicts the random traffic at time + 1 given the traffic of the event-driven channels. Bandwidth is allocated for the random and event-driven channel based on the following optimization as in [14]. The sampling period of the NCS is varied to optimize bandwidth allocation. Even for a linear timeinvariant plant, the closed-loop NCS is not time-invariant and the placement of its eigenvalues inside the unit circle is no longer sufficient for stability. Zhou and Zhao provided the following result regarding the stability of the perturbed discrete linear time-varying system with state matrix ( ) and perturbation matrix Δ ( ) [17].
( + 1) = ( ( ) + Δ ( )) ( ), = 0, 1, 2, … where ( ), ( ) ∈ ℛ × Theorem 1 [17]: Assume that the nominal discrete linear time-varying system with state matrix ( ) is uniformly exponentially stable. If there exist a positive constant and a sufficiently small constant such that Δ ( ) satisfies then, the perturbed discrete linear time-varying system system (8) is uniformly exponentially stable. Based on the above theorem, we obtain a condition for the stability of a linear time-invariant NCS with time-varying sampling period. We assume that the state matrix has geometric multiplicity equal to its algebraic multiplicity. Theorem 2 : If the linear time-invariant system with digital control is closed-loop stable with state feedback control gain and sampling period for the time driven channel of the related control loop ℎ , then the system is uniformly exponentially stable with time-varying sampling period in a bounded interval The state-transition matrix and the input matrix can be written as For a zero eigenvalue, we have the term ℎ ( ), which can be handled similarly to what we show below. Perturbation in the sampling period will result in the perturbation matrix For a sampling period varying in a bounded interval [ , ], we have Because the nominal dynamics are stable The condition of Theorem-1 [17] becomes

RESULTS AND DISCUSSIONS
A DC motor position control example adapted from [23,24] is simulated to compare our results with fixed bandwidth allocation. The DC motor transfer function is; A feedback controller is designed to achieve desired transient response. An upper bound for the sampling period of the time-driven channels ℎ , is necessary for obtaining satisfactory NCS performance and is calculated based on the system dynamics. The desired response of the system must have percentage overshoot less than 20% and settling time less than 0.2 s. For 20% overshoot, required damping ratio is approximately 0.46. The natural frequency of the closed loop system is calculated based on the specifications as = 48.73rad/s. The undamped natural frequency should be slower than the packet arrival to the controller, to obtain the required sampling frequency for the NCS, is selected as = , where a scale factor selected to reduce aliasing. Assuming = 1s, we selected = 15 to obtain ℎ = 2 ≅ 11 ms. since for the network is 0.1 s, ℎ = 1.1 ms. Based on hardware limitation on sampling rate, we select ℎ = 0.6 ms. For the simulation, frequency-division multiple access (FDMA) network channel is used in NCS as in [25]. The FDMA network channel has a capacity of 495 Mbps and packet size is 100 bytes. Thus, the total network capacity is 495 pac/0.1s. The traffic in the three random channels is predicted and the sensor and the controller channels generate 100pac/0.1s to satisfy required transients response. Random networks occupy 80% of the bandwidth of the network while time-driven sensor and eventdriven controller nodes occupy the remaining 20%. Fixed sampling period of the time-driven sensor channel is choosen as 1 s. If the allocated bandwidth for random channels is high, the allocated bandwidth for the controller and the sensor channels may become insufficient and this may cause delay or package drop which deteriorates the performance of the NCS as shown in Figure 3.

Figure 3. NCS output without bandwidth allocation
Assume that, three random nodes which generate random traffic and two deterministic nodes which are the time-driven sensor node and event-driven controller node are sharing the network in Figure 2. The HMM model proposed in Section 2 with 125 states for three random channels with the channel conditions { , , , , }. An adaptive (HMM-1) is constructed by selecting an appropriate window size = 10 and the Viterbi algorithm predicts future traffic volume for random channels. Since rest of the channels are deterministic and they have fixed network parameters, their traffic can be calculated from the parameters. The controller contains an event-driven task and every instances, a sample arrives over the network from the sensor node. The network traffic model used in the paper is offline, however since HMM model of the network traffics are modelled, if priory network traffic in the channels are known in the selected window size this model will work for online network traffic model also.
To evaluate the effect of the number of traffic classes of the HMM model, we construct an HMM model with 27 states for three random channels with three conditions{ , , }, dubbed (HMM-2). The latter is used to predict the next random traffic volume for comparison to HMM-1.
The traffic in the controller to actuator channel is 100 pac/0.1sec. The controller computes a control signal after receiving the packet, then the packet that includes control signal is sent to the event-driven actuator node. After predicting the traffic of the random channel at time + 1, we have values for all network traffic that is used for bandwidth allocation. The bandwidth allocation is ( + 1) = [̃( + 1), , 100], where is to be calculated using the convex optimization of (8).
After allocating bandwidth optimally, the sampling period for the time-driven channel is obtained by employing (9). The sampling period of the sensor to controller channel ℎ ( + 1) is calculated using (10). For a satisfactory output response, the sampling period of the time-driven channel is adjusted to eleminate channel congestion.
At first, fixed network parameters without bandwidth allocation and sampling rate adjustment is used. Network congestion occurs as the traffic of some channels exceeds their capacity. Figure 3 shows that the closed-loop system response is oscillatory because network congestion causes packet loss and time delay.
The available bandwidth is allocated depending on the demand of each channel. Bandwidth allocation for the random and the event-driven controller channel is achieved using 1 optimization. Because the total capacity of Network is limited, insufficient bandwith can be allocated for the time-driven sensor channel and network congestion may occur for the time-driven sensor channel. Its sampling rate is adjusted to satisfy its allocated bandwidth capacity subject to stability and performance constraints. Figure 4 indicates that the sampling period of the sensor channel stays in the interval [0,0.25] for both HMM-1 and HMM-2. The sampling period is assigned according to the allocated bandwidth for the time-driven sensor channel. The sampling period never exceeds the prescribed upper and lower bounds [0.6, 1.1] ms. Figure 5 and Figure 6 show that the bandwidth allocation using both HMM-1 and HMM-2 provides satisfactory responses. However, Figure 6 shows that the response of HMM-2 includes sharp perturbations because of the large prediction error and limited sampling period interval. Figure 7 shows the  [24]. The network scheduling varies based on the dynamic sampling period assignment and bandwidth allocation.  Figure 9 verifies that HMM-1 provides satisfactory bandwidth allocation and meets traffic demand better than HMM-2.  Table 1 show the predicted random traffic for HMM-1 and HMM-2, respectively. The random traffic in the network is predicted by employing HMM and the Viterbi algorithm and used for bandwidth allocation. The figures show that, prediction errors for the states are lower for HMM-1. This provides more efficient bandwidth allocation and sampling period scheduling based on the allocation of bandwidth for the sensor channel. Figure 13 and Table 1 reveal that an adequate but small window size provides better state prediction. An excessive window size results in larger state prediction errors because of changes in network dynamics over time that make older data inapplicable. Increasing the number of network states that define the traffic condition reduces the prediction error.

CONCLUSION
This paper provides a new approach for bandwidth allocation for NCS. An adaptive HMM with 5 traffic classes is constructed for random traffic and the traffic is estimated utilizing the Viterbi algorithm. The HMM parameters are updated based on the traffic data for a window of specified width to account for changes in traffic conditions. l 1 Optimization method is employed to allocate bandwidth for the time-driven sensor channel in NCS. The sampling period of the time-driven sensor channels is adjucted to mitigate network traffic and avoid packet loss and delay but kept within bounds dictated by performance requirements and bandwidth limitations. We present conditions for the stability of the resulting time-varying system. A second HMM network with only 3 traffic classes, as is common in the literature, is also designed for comparison to the 5-class HMM. Simulation results where the 5-class HMM outperforms the 3-class HMM demonstrate the importance of having a sufficient number of traffic classes for the HMM. Results for different data window widths demonstrate the importance of selecting an appropriate window width to capture the current traffic dynamics of the network. Future work will apply the methodology to more complex NCS and exploit knowledge of random traffic statistics for bandwidth allocation.