IEEE INTERNET OF THINGS JOURNAL, VOL. 6, NO. 2, APRIL 2019 3743 Machine Learning-Based Bandwidth Prediction for Low-Latency H2M Applications Lihua Ruan , Graduate Student Member, IEEE, Maluge Pubuduni Imali Dias, and Elaine Wong , Senior Member, IEEE Abstract—Human-to-machine (H2M) communications in emerging tactile-haptic applications are characterized by stringent low-latency transmission. To achieve low-latency transmissions over existing optical and wireless access networks, this paper proposes a machine learning-based predictive dynamic bandwidth allocation (DBA) algorithm, termed MLP-DBA, to address the uplink bandwidth contention and latency bottleneck of such networks. The proposed algorithm utilizes an artificial neural network (ANN) at the central office (CO) to predict H2M packet bursts arriving at each optical network unit wireless access point (ONU-AP), thereby enabling the uplink bandwidth demand of each ONU-AP to be estimated. As such, arriving packet bursts at the ONU-APs can be allocated bandwidth for tranmission by the CO without having to wait to transmit in the following transmission cycles. Extensive simulations show that the ANN-based prediction of H2M packet bursts achieves >90% accuracy, significantly improving bandwidth demand estimation over existing prediction algorithms. MLP-DBA also makes adaptive bandwidth allocation decisions by classifying each ONU-AP according to its estimated bandwidth, with results showing reduced uplink latency and packet drop ratio as compared to conventional predictive DBA algorithms. Index Terms—Artificial neural network (ANN), low latency, machine learning, predictive bandwidth allocation. I. I NTRODUCTION HE emerging Internet-of-Things and Tactile Internet paradigms are instigating the evolution of machine-tomachine communications toward real-time and remotely controlled human-to-machine (H2M) communications. Effective H2M communications, such as responsive autonomous systems, immersive edutainment, and proactive healthcare [1], rely on low-latency (in milliseconds order) delivery of H2M packets from/to human operators and the corresponding remotely-controlled devices/robots [2], [3]. An example network architecture that supports H2M communications is illustrated in Fig. 1 [1]–[4]. As illustrated, bidirectional transmission of H2M packets traverses the different segments of the Internet including optical access networks, the integrated optical network unit and wireless access points (ONU-APs), T Manuscript received September 3, 2018; revised October 9, 2018 and November 10, 2018; accepted December 25, 2018. Date of publication January 1, 2019; date of current version May 8, 2019. (Corresponding author: Lihua Ruan.) The authors are with the Electrical and Electronic Engineering Department, University of Melbourne, Melbourne, VIC 3010, Australia (e-mail: ruanl@student.unimelb.edu.au; diasm@unimelb.edu.au; ewon@unimelb.edu.au). Digital Object Identifier 10.1109/JIOT.2018.2890563 Fig. 1. Network architecture that supports H2M communications. and wireless local area networks (WLANs). Specifically, stateof-the-art literature has reported that stringent end-to-end latency (within 1∼10 ms) must be adhered to in supporting remote and real-time control/actuation/haptic feedback delivery over such heterogeneous wireless and optical access networks [1]–[4]. To address this latency challenge, we have presented in [4], [7], and [8], that strategically placing control servers at the central office (CO) closer to end users can effectively expedite the delivery of H2M packets, and consequently the overall response and actuation of devices/robots. In this regard, bandwidth resource allocation decision made by the CO for delivering H2M traffic over access networks is critical to improve the latency performance. Similar to the burstiness of Internet traffic [12], experimental studies in [9]–[11] have revealed that H2M traffic generated in H2M applications are also bursty with alternate ON and OFF intervals. For example, during an ON interval, consecutive packets generated by end devices/robots are wirelessly transmitted through the WLANs and aggregated at the ONU-APs, whereas no packet is generated in the OFF interval. While downlink transmissions in Fig. 1 are typically broadcast, the latency bottleneck mainly lies in the uplink transmission due to bandwidth contention amongst multiple ONU-APs. Specifically, uplink bandwidth allocation is mainly achieved by implementing dynamic bandwidth allocation (DBA) algorithms. In a typical DBA algorithm, an ONU-AP demands bandwidth by appending a REPORT message to its packet transmissions. Accordingly, the CO grants bandwidth to the ONU-AP through a GATE message in a round-robin manner [12]. The time interval between consecutive transmissions of an ONU-AP is known as a polling cycle. The uplink latency focused in this paper hence refers to the time upon a packet arriving at an ONU-AP until it reaches CO. Classic DBA algorithms including fixedservice DBA, exhaustive-service DBA, limited-service DBA, c 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. 2327-4662 See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. Authorized licensed use limited to: Microchip Tech Inc. Downloaded on January 24,2022 at 13:12:05 UTC from IEEE Xplore. Restrictions apply. 3744 IEEE INTERNET OF THINGS JOURNAL, VOL. 6, NO. 2, APRIL 2019 and excessive-service DBA have been comprehensively studied and compared in [12]–[14]. Results show that these DBAs exhibit high uplink latency when traffic is bursty. This is partly attributed to the fact that arriving packets at the ONU-AP need to be reported first by the ONU-AP and then subsequently granted bandwidth for transmission in the next polling cycle. Moreover, packets buffered at highly loaded ONU-APs may need to wait for multiple cycles before bandwidth is granted for their transmission. Due to the marginal difference in uplink latency performance between the above classic DBAs, the limited-service DBA algorithm, an algorithm that prevents bandwidth monopolization by heavily-loaded ONU-APs, is commonly adopted as the baseline DBA [12]. On the other hand, predictive DBAs typically provide additional bandwidth to each ONU-AP on top of the requested amount. This additional bandwidth, termed BWpred in this paper, is typically estimated by predicting the amount of arriving packets (in bytes) at each ONU-AP during each polling cycle [15]. Predictive DBAs have the potential to reduce uplink latency as additional packets can be transmitted in the same polling cycle without their bandwidth having to be reported by the ONU-APs. However, when it comes to predicting the bandwidth of bursty traffic, this remains an open challenge. The effectiveness of a predictive DBA is dependent on its prediction accuracy. Since accurately predicting the bandwidth of bursty traffic will reduce uplink latency [14], it is imperative that the bandwidth prediction of bursty traffic is investigated in order to reduce the uplink latency of networks that support H2M traffic. Further, the bandwidth allocated by the CO should be in accordance with the uplink traffic characteristics and load. For highly bursty H2M traffic, the ability of DBAs in allocating bandwidth during bursty intervals and preserving bandwidth during long-idle intervals is key to reducing the uplink latency [9]. In light of the above, in this paper, we exploit the use of an artificial neural network (ANN) at the CO to predict the ON and OFF status of bursty H2M traffic arriving at each ONU-AP. Prediction of the ON/OFF status consequently enables the evaluation of the uplink bandwidth demand of each ONU-AP. In justifying prediction performance, we present a comparative study of ANN versus existing bursty traffic prediction methods used in predictive DBAs. Through simulations, we first show that our trained ANN can achieve superior accuracy (>90%) in predicting the ON and OFF status of ONU-APs, and therefore, effectively improving the accuracy of the bandwidth demand estimation. The proposed ANN prediction is then implemented at the CO to facilitate bandwidth allocation decisions through the proposed machine learning-based predictive DBA (MLP-DBA). In contrast to conventional DBA/predictive DBA algorithms that grant requested/estimated bandwidth to all ONU-APs, MLP-DBA classifies each ONU-AP based on: 1) the predicted ON and OFF status of ONU-APs and 2) the estimated bandwidth demand. The CO then adaptively allocates bandwidth to each ONU-AP. In this paper, we show that with ANN prediction, the proposed MLP-DBA successfully improves uplink latency performance and reduces the packet drop ratio as compared to the baseline limited-service DBA and existing predictive DBA algorithms. The rest of this paper is organized as follows. In Section II, we introduce state-of-the-art predictive DBA algorithms and the motivations behind applying ANN in a DBA. Our ANN-based ON/OFF status prediction is proposed in Section III. Section IV details the proposed MLP-DBA algorithm with analytical and simulation results and discussions presented in Section V. A summary of the main contributions of this paper is presented in Section VI. II. E XISTING P REDICTIVE DBA A LGORITHMS Early predictive DBA algorithms such as constant and linear credit DBAs typically allocate a constant BWpred to ONU-APs in each polling cycle irrespective of traffic load. Hence, these algorithms, frequently termed as static predictive DBAs, provide only marginal latency improvements over nonpredictive DBAs if the considered traffic is bursty [12]. By comparison, statistical predictive DBAs reported in [16]–[18] use arithmetic averaging, exponential smoothing, and Bayesian estimation to evaluate BWpred by estimating the packet interarrival time. In terms of bursty traffic, the latency performance of these DBAs in [16]–[18] were shown to fluctuate due to their limitation in predicting instantaneous traffic variations [4]. On the other hand, the predictive algorithm in [19], termed mostrecently (MR) DBA, uses the MR received packets from the previous polling cycle to estimate BWpred . Uplink latency was shown to be reduced as compared to the limited-service DBA. Further, the limited sharing DBA with traffic prediction (LSTP-DBA) [20] uses a 4-order autoregressive model, i.e., counting on the arrival packets in the previous four polling cycles, to estimate BWpred . Simulation results show that LSTP-DBA outperforms classic DBA algorithms regarding the latency performance. Yet, the prediction accuracy, a critical criterion for evaluating the prediction performance [14], in MR- and LSTP-DBAs, remains to be investigated. Most recently in [21], a machine learning technique, k-nearest neighborhood (kNN), was used to evaluate BWpred for video streams. The proposed algorithm, named data mining forecasting DBA, estimated BWpred by averaging the received packets in k number of past polling cycles that have similar durations to the current cycle. We verified in a preliminary study [22] that even though kNN-based prediction was effective in predicting constant video flows, it was less effective in bursty scenarios, and results in large prediction errors. The prediction performance of the algorithms mentioned above depends on the applications supported and also the selection of traffic features used in making the prediction, e.g., interarrival time, received packets, or polling cycle durations, etc. As such, the prediction accuracy for busty traffic is still to be investigated. Compared to the predictive DBAs described above that directly predict the value of BWpred using past received packets, in this paper, we propose to exploit an ANN located at the CO that is able to input multiple traffic features to predict the ON/OFF status of each ONU-AP, such that BWpred of the ONU-AP can be estimated with high accuracy for bursty traffic. Then, using MLP-DBA, the CO adaptively classifies Authorized licensed use limited to: Microchip Tech Inc. Downloaded on January 24,2022 at 13:12:05 UTC from IEEE Xplore. Restrictions apply. RUAN et al.: MACHINE LEARNING-BASED BANDWIDTH PREDICTION FOR LOW-LATENCY H2M APPLICATIONS 3745 Fig. 2. Operational traffic flow diagram of a typical predictive DBA operation algorithm. ONU-APs and allocates bandwidth accordingly. Note that our preliminary work in [22] presents the advantages of using an ANN in bandwidth prediction and allocation over the existing use of kNN for H2M applications. This paper on the other hand, details the proposed ANN prediction model, supervised training process, prediction performances, and comprehensive validates the effectiveness of the proposed MLP-DBA in addressing the challenges in conventional limited-service DBA, MR-DBA, and LSTP-DBA algorithms. III. A RTIFICIAL N EURAL N ETWORK -BASED P REDICTION A. Traffic Flow in Predictive DBA Fig. 2 illustrates the operational flow diagram of a typical predictive DBA. Let us denote Tpoll (i, j) as the duration of the ith polling cycle of ONU-AP j. As illustrated, in the ith polling cycle, ONU-AP j transmits k(i, j) bytes, or equivalently n(i, j) number of packets, in the granted transmission duration and reports the remaining queue length, denoted as BWreq (i, j), in the REPORT message to the CO. Hence, BWreq (i, j) is the requested bandwidth by ONU-AP j. The CO then allocates bandwidth, BWdem (i+1, j), by sending a GATE message that indicates the next transmission start time and duration of ONU-AP j. The duration of the (i+1)th polling cycle, Tpoll (i + 1, j), is updated at the CO. Upon receiving the GATE message from the CO, ONU-AP j waits for the next transmission start time which it then transmits packets for a duration equivalent to BWdem (i + 1, j). As shown in Fig. 2, while waiting for the transmission start time in the (i + 1)th polling cycle, an amount of a(i + 1, j) bytes is received by ONU-AP j for uplink transmission. To reduce uplink latency, the CO predicts a(i + 1, j), or the equivalent bandwidth BWpred (i + 1, j). The total bandwidth, BWdem (i + 1, j), that is allocated by the CO to ONU-AP j, should therefore be calculated as follows: BWdem (i + 1, j) = BWreq (i, j) + BWpred (i + 1, j). (1) With BWpred (i+1, j) allocated, a(i+1, j) bytes arriving during the (i+1)th polling cycle can be transmitted within the polling cycle without needing to be reported to the CO, thereby reducing latency. However, the exact a(i+1, j) is not known a priori and so accurately predicting a(i + 1, j), i.e., BWpred , is critical to improving uplink latency performance. Let us now denote the prediction error as follows: (2) e(i + 1, j) = BWpred (i + 1, j) − a(i + 1, j). Fig. 3. Schematic of the proposed ANN architecture to predict the ONU/OFF status (y) of the bursty traffic at ONU-AP j. Note that as how long a packet will wait at the ONU-APs before transmission depends on DBA operations, investigation into bandwidth allocations for packets aggregated at ONU-APs is our focus in the current work. In a heterogeneous access networks, bursty network traffic characterized by ON/OFF intervals has been rigorously studied in [26]–[28]. In H2M experiments, human operations have also shown similar ON/OFF packet bursts [9]. In this paper, we adopt the typical ON/OFF traffic model [26] widely used to simulate the aggregations of H2M traffic. The sensitivity of the proposed ANN prediction to different levels of burstiness and traffic model is evaluated anticipating the emerging applications. B. Artificial Neural Network As introduced earlier, when considering bursty traffic, it is critical for the CO to identify if an ONU-AP is currently in an ON interval, i.e., the ONU-AP is receiving packets aggregated from its WLAN, or otherwise in an OFF interval, i.e., an idle interval with no arriving packet bursts. Erroneously allocating additional bandwidth to ONU-APs that are in their OFF intervals increases polling cycle time, and thereby increases the uplink latency of the packets at those ONU-APs. Hence, we use an ANN to predict the ON/OFF status of ONU-AP j and estimate BWpred (i + 1, j) from this prediction. Fig. 3 illustrates a schematic of our proposed ANN comprising an input layer, an output layer, and some hidden layers in between. Each layer is composed of neuron units, which are nonlinear functions that map the associated inputs to an output. Specifically, when receiving the REPORT from ONU-AP j in the ith polling cycle, the CO uses available traffic features, as listed in Table I to predict the ON/OFF burst status of ONU-AP j in the (i + 1)th polling cycle. With x(i, j) = {k(i, j), n(i, j), a(i, j), BWreq (i, j), Tpoll (i, j), Tpoll (i + 1, j)} as the input traffic features to the ANN in the ith polling cycle, the target output y is denoted in (1) as follows: 1, ONU-AP j is ON in (i + 1)-th polling cycle y(i + 1, j) = 0, ONU-AP j is OFF in (i + 1)-th polling cycle. (3) Authorized licensed use limited to: Microchip Tech Inc. Downloaded on January 24,2022 at 13:12:05 UTC from IEEE Xplore. Restrictions apply. 3746 IEEE INTERNET OF THINGS JOURNAL, VOL. 6, NO. 2, APRIL 2019 TABLE I T RAFFIC F EATURES C OLLECTED IN THE ith P OLLING C YCLE An ANN learns by iteratively adjusting its weights and bias associated with the neurons to a yield desired output. It has a proven capability in addressing curve fitting, optimization, pattern recognition, and classification problems [23]. Here, based on six input traffic features, our proposed ANN devolves into the classification of an ON (y = 1) or OFF (y = 0) status. Recall that BWpred (i + 1, j) is the estimated bandwidth of a(i + 1, j) at ONU-AP j. Considering lastmile compatibility [20], [24] and the ON/OFF status y(i + 1, j) predicted by the ANN, BWpred (i + 1, j) can be estimated using BWpred (i + 1, j) ≈ y(i + 1, j)Tpoll (i + 1, j)RWLAN . (4) In (2), RWLAN is the data rate of the WLAN associated with ONU-AP j. Once the ON/OFF status y(i + 1, j) is predicted, and the BWpred (i, j) is estimated, the CO evaluates the total bandwidth demand, BWdem (i + 1, j), of ONU-AP j in (i + 1)th polling cycle as follows: BWdem (i + 1, j) = BWreq (i, j) + y(i + 1, j)Tpoll (i + 1, j)RWLAN . (5) C. Supervised Training Prior to using the ANN to predict the ON/OFF status of ONU-APs, we must first train the ANN using training sets collected during the training phase of network operation. A training set, S, is represented as {(x(i, j), y(i + 1, j))| for i = 1, 2 . . . , N, j = 1, 2 . . . , M} with x(i, j) as the traffic features collected by the CO in the ith polling cycle and y(i+1, j) as the labeled output. The parameter N is the total number of polling cycles, and M is the number of ONU-APs. Here, we generate training sets using event-driven packet-level simulations of 16and 32-ONU-AP optical access networks. The data rate of the optical access network and WLANs are 1 Gb/s and 100 Mb/s, respectively. Traffic features from 250 000 polling cycles under various network loads at each ONU-AP are collected and then is utilized for supervised training. The Hurst parameter in the traffic model is a primary indicator for self-similarity and long-range dependence (LRD) of network traffic aggregated from diverse applications [26]. A Hurst parameter of 0.8, i.e., H = 0.8, commonly utilized by existing traffic profiles is chosen here to simulate packet bursts in the WLANs [12]. The sensitivity of the trained ANN is presented in the following section, considering a variation in Hurst parameter as well as an exponential traffic model [12] for comparison. After supervised training with the collected training set, the final architecture of the ANN is shown in Fig. 3, whereby in addition to the input and output layers, there is one hidden layer of ten neurons. The optimal weight matrix for each (a) (b) Fig. 4. ON / OFF status prediction performance. (a) Prediction accuracy. (b) ROC (traffic load = 0.6). layer is determined using Gradient descent method [25]. Since prediction accuracy does not improve with additional layers and neurons, this architecture is used as a final architecture in subsequent investigations. Note that once the training phase is complete, the CO only needs to store the weight matrix to map the input traffic features to output the predicted ON/OFF status (3), estimate BWpred (4), and evaluate (5). To verify the predict performance of the trained ANN, we use a test set, T, with another 250 000cycle traffic features, to investigate the accuracy of its ON/OFF status prediction. Fig. 4(a) plots the ON/OFF prediction accuracy for the 16 and 32 ONU-AP networks as a function of aggregated network load, highlighting that the trained ANN achieves high prediction accuracy for network loads from 0.1 to 1. The prediction accuracy is >96% under light and high network loads due to the ONU-APs having longer idle (OFF) and bursty (ON) intervals, respectively. By comparison, the prediction accuracy decreases under moderate traffic load, i.e., 0.4∼0.7. This behavior can be attributed to the frequent alternation of ON and OFF intervals. To investigate further, we plot the receiver operating characteristic (ROC) curves at a network load of 0.6 [Fig. 4(b)]. Here, the true positive rate is the percentage of correct ON predictions in all ON samples, and the false positive rate is the percentage of mispredicting OFF over all OFF samples. Note that the curves approach (0,1), indicating that the trained ANN achieves effective prediction for both ON and OFF status. Also, the trained ANN achieves better prediction performance for 32-ONU-AP network, as also indicated in Fig. 4(a). Considering the same network load, each individual ONU-AP in the 32-ONU-AP network contributes to a lower network load than an individual ONU-AP in a 16-ONU-AP network. Authorized licensed use limited to: Microchip Tech Inc. Downloaded on January 24,2022 at 13:12:05 UTC from IEEE Xplore. Restrictions apply. RUAN et al.: MACHINE LEARNING-BASED BANDWIDTH PREDICTION FOR LOW-LATENCY H2M APPLICATIONS 3747 (a) (a) (b) (b) Fig. 5. Evaluation of sensitivity to training set size and traffic model variation. (a) Training set size variation. (b) Traffic model variation. Fig. 6. Prediction performance comparison with MR-DBA and LSTP-DBA. (a) 16 ONU-AP. (b) 32 ONU-AP. In Fig. 5, we investigate the dependency of the ON/OFF prediction accuracy on the training set size, Hurst parameter variation in the Pareto traffic model, and an exponential traffic model for a 32-ONU-AP. In Fig. 5(a), results for various network loads at 0.3, 0.6, and 0.9 are shown for illustrative purposes. As can be observed, increasing the training set size improves prediction accuracy. Further, the prediction accuracy varies under different network traffic loads. As shown in Fig. 5(a), network loads of 0.3 and 0.9 require a smaller training set to reach >90% accuracy whereas a larger training set is required for the network load of 0.6 to achieve the same prediction accuracy. For the network loads considered, prediction accuracy is low when the training set is small due to an inadequate number and imbalance of ON/OFF samples. Specifically, as each time of packet burst at each ONU-AP differentiates significantly, a small training set only containing features of a few ON/OFF intervals cannot be well generalized to predict a test set. When samples are enough to include features from enough ON/OFF alternations at all 32 ONU-APs, a clear raise in prediction accuracy can be observed. Results show that overall, across the three network loads, a training set size with around 104 samples is sufficient to train the ANN whereby any further increase in training set size would not yield further improvement in prediction accuracy. Fig. 5(b) shows the sensitivity of the trained ANN. The trained ANN is first applied to predict the ON/OFF status of ONU-APs when the Hurst parameter varies from 0.6 to 0.9. Note that 0.5 < H < 1 describes the long-term auto-correlations, i.e., LRD, of the aggregated traffic [29]. A larger H represents stronger LRD and traffic burstiness, indicating that consecutive long ON intervals are more likely to occur. Results in Fig. 5(b) shows that the trained ANN achieves >95% prediction accuracy regardless of H variations in traffic. In comparison with LRD traffic, the trained ANN is also applied to traffic characterized by short-range dependence (SRD) with exponential ON/OFF intervals. As can be viewed in Fig. 5(b), the prediction accuracy (green bar) reduces, but >90% prediction accuracy can still be achieved. Further, when retraining the ANN corresponding to the SRD traffic, the prediction accuracy (yellow bar) is effectively improved to 98%. Results in Fig. 5(b) validate the robustness of the trained ANN. D. Bandwidth Prediction With the trained ANN predicting the ON/OFF status of all ONU-APs using (3), and estimating BWpred using (4), we evaluate the average prediction error using Eavg = M N M N 1 1 e(i, j) = BWpred (i, j) − a(i, j). NM NM j=1 i=1 j=1 i=1 (6) Fig. 6(a) and (b) plots Eavg for a 16- and 32-ONU-AP network, respectively, and compares the results obtained using our proposed ANN-based prediction method to that obtained using the MR-DBA and LSTP-DBA. Fig. 6 shows that Eavg increases with traffic load and that our proposed ANN-based prediction has the best performance, especially under heavy traffic loads. Eavg of the 32-ONU-AP network is about half of that of the 16-ONU-AP network. Again, this is attributed to the lower traffic load per individual ONU-APs in 32-ONUAP network when considering the same aggregated network load. Authorized licensed use limited to: Microchip Tech Inc. Downloaded on January 24,2022 at 13:12:05 UTC from IEEE Xplore. Restrictions apply. 3748 IEEE INTERNET OF THINGS JOURNAL, VOL. 6, NO. 2, APRIL 2019 TABLE II P RINCIPLE OF ONU-AP C LASSIFICATION AND MLP-DBA BANDWIDTH A LLOCATION min Fig. 7. Illustration of MLP-DBA procedure with three ONU-APs. IV. MLP-DBA A LGORITHM The CO in an optical access network will typically allocate a bandwidth equivalent to min{BWdem , BWmax } to ONU-APs during each polling cycle. The parameter BWmax is the maximum bandwidth that can be allocated to an individual ONU-AP and BWdem = BWreq + BWpred as given in (1). When ONU-APs are heavily loaded with long accumulated packet queues, BWreq exceeds BWmax . For heavily loaded ONU-APs, reducing the time interval between consecutive transmissions and increasing the allocated bandwidth would reduce the uplink latency of the arriving packets, but this must be done without impacting the less heavily loaded ONU-APs. In order to expedite transmission of arriving packets, MLP-DBA enables the CO to classify ONU-APs based to their BWdem , and allocate bandwidth accordingly, as described below. Upon receiving a REPORT message from an ONU-AP, the CO predicts the ON/OFF status of the ONU-AP, i.e., y using the trained ANN, and estimates BWpred and BWdem as introduced in Section III. Then, the CO classifies each ONU-AP into one of 3 different classes following the principle listed in Table II. 1) If according to the train ANN, the status of an ONU-AP is ON in the next polling cycle, i.e., y = 1, and meanwhile BWdem < BWmax , the ONU-AP will be classified as Class A. For such an ONU-AP, the CO will allocate through a GATE message a bandwidth equivalent to BWdem = BWpred + BWreq for transmission in the next polling cycle, allowing arriving packets to be transmitted without having to be reported to the CO and without having to wait for the next transmission time. 2) If the status of the ONU-AP is predicted to be OFF, i.e., y = 0, with BWdem < BWmax , it will be classified as Class B. The CO will allocate a Class-B ONU-AP a bandwidth that is equivalent to BWreq since no packet is expected to arrive during its OFF interval. 3) Heavily loaded ONU-AP with a BWdem > BWmax is classified as Class C. For such an ONU-AP, the CO will first allocate BWmax through a GATE message. Further, to alleviate the number of queued packets, an additional GATE message is generated for the ONU-AP by the CO such that an additional transmission opportunity is given to the ONU-AP before scheduling the next polling cycle. To explain the operation of MLP-DBA for Class-A, B, and C ONU-APs more clearly, we present the MLP-DBA operation in Fig. 7 with 3 ONU-APs for illustrative purposes. As shown in Fig. 7, in the (i − 1)th polling cycle, ONU-AP 1, 2, and 3 are classified as Class A, C, and B, respectively. As such, in responding to their REPORTs, the CO first grants BWdem to ONU-AP 1, 2, and 3 in the ith polling cycle, respectively. Moreover, since ONU-AP 2 is classified as Class C, at the beginning of the ith polling cycle, an additional GATE message is generated for ONU-AP 2 by the CO such that an additional transmission opportunity is given to ONU-AP 2 before scheduling the (i + 1)th polling cycle. As such, the transmission interval for ONU-AP 2 is reduced as presented in the figure. As ONU-AP 3 is OFF with no packet arrivals expected, the additional transmission opportunity interval does not affect ONU-AP 3. Note that additional GATEs are generated and send at the beginning of each polling cycle based on the classification results in the previous cycle. As such, the CO does not respond to the REPORTs received during additional transmission opportunity intervals. The CO repeats this procedure during each polling cycle for all ONU-APs. It is worth noting that MLP-DBA algorithm considers bursty traffic pattern and reduces latency by adaptively allocating bandwidth to ONU-APs in ON intervals. As such, it can be applied to bursty Authorized licensed use limited to: Microchip Tech Inc. Downloaded on January 24,2022 at 13:12:05 UTC from IEEE Xplore. Restrictions apply. RUAN et al.: MACHINE LEARNING-BASED BANDWIDTH PREDICTION FOR LOW-LATENCY H2M APPLICATIONS Algorithm 1 MLP-DBA: Allocation at the CO Pseudocode of Bandwidth tscheduled — time up to which the uplink channel has been scheduled RRT — round-trip transmission time T g — guard time T process — time for processing a GATE message at an ONU-AP T REPORT — duration of a REPORT message δ(i, j) ∈ {A, B, C} — class of ONU-AP j in ith polling cycle tstart (i, j)/tadd_start (i, j) — the start time of granted timeslot for ONU-AP j in the ith polling cycle tend (i, j)/tadd_end (i, j) — the end time of granted timeslot for ONU-AP j in the ith polling cycle RPON — data rate in the optical network RWLAN — data rate in the WLAN BWadd_grant (i, j) — the granted bandwidth for ONU-AP j in the additional opportunity if an additional GATE is generated for ONU- AP j in the ith polling cycle ANN(x) — trained neural network with input feature vector x The operation for ONU-AP j at the CO at the beginning of ith polling cycle (repeat for all i and j): 1 { 2 if δ(i, j) = C 3 // schedule the start time in the ith additional cycle 4 tadd_start (i, j) = min{LocalTime + RRT/2 + Tprocess , 5 tscheduled – RRT/2 – T process }; 6 BWadd_grant (i, j) = BWmax ; 7 tadd_end (i + 1, j) = tadd_start (i, j) + BWmax /RPON + 8 T REPORT + Tguard ; 9 generate and send a GATE indicating tstart (i + 1, j) 10 and BWadd_grant (i, j); 11 tscheduled = tadd_end (i, j); // update scheduled time 12 else 13 BWadd_grant (i, j) = 0; 14 end 15 } The operation for ONU-AP j at the CO upon receiving the REPORT in the ith polling cycle (repeat for all i and j): 16 { 17 // schedule the start time in the (i + 1)th polling cycle 18 tstart (i + 1, j) = min{LocalTime + RRT/2 + Tprocess , tscheduled 19 −RRT/2 − Tprocess }; 20 //ANN-based prediction and classification 21 xi , j = {k(i, j), n(i, j), a(i, j), BWreq (i, j), TPOLL (i, j), 22 TPOLL (i + 1, j)}; 23 y = ANN(xi , j); 24 BWpred (i + 1, j) = yRWLAN tstart (i + 1, j) − yRWLAN tstart (i, 25 j)tstart (i, j); 26 update δ(i + 1, j); 27 // following the limited-service principle, the granted BW 28 is bounded by BWmax 29 BWgrant (i + 1, j) = min{BWreq (i, j) + BWpred (i + 1, j)− 30 BWadd_grant (i, j), BWmax }; 31 // update time schedule and generate a GATE message 32 tend (i + 1, j) = tstart (i + 1, j) + BWgrant (i + 1, j)/RPON + 33 TREPORT + Tguard ; 34 generate and send a GATE indicating tstart (i + 1, j) and 35 BWgrant (i + 1, j); 36 tscheduled = tend (i + 1, j); // update scheduled time 37 } The operation for ONU-AP j at the CO upon receiving the REPORT in the ith additional opportunity (repeat for all i and j): 38 { 39 xi,j = {k(i, j), n(i, j), a(i, j), BWreq (i, j), TPOLL (i, j), 40 TPOLL (i + 1, j)}; // note that features are collected from 41 the additional cycle 42 y = ANN(xi,j ); 43 update δ(i + 1, j); 44 } traffic with different levels of burstiness such as that studied in Fig. 5(b). The pseudocode of the MLP-DBA algorithm is presented in Algorithm 1. 3749 TABLE III N ETWORK AND P ROTOCOL PARAMETERS V. P ERFORMANCE E VALUATION To verify the effectiveness of MLP-DBA, we implemented packet-level simulations (MATLAB) of an optical access network which parameters are listed in Table III. The bursty ON and OFF intervals follows Pareto distribution with a Hurst parameter of 0.8. We analyze the ONU-AP classifications during MLP-DBA operation and compare the uplink latency and packet drop ratio performances of MLP-DBA with/without ONU-AP classification, MR-DBA, LSTP-DBA, and the baseline limited-service DBA algorithms. In the case of MLPDBA without classification, BWdem is estimated using the proposed ANN and (3–5) but bandwidth allocation follows the conventional DBA operation whereby min{BWdem , BWmax } is allocated to all ONU-APs. In contrast, MLP-DBA with classification utilizes the ANN to predict the ON/OFF status and estimate BWdem of each ONU-AP and based on these results, classify and allocate bandwidth to each ONU-AP accordingly. A. Uplink Latency and Packet Drop Ratio Fig. 8(a) and (b) verify the effectiveness of MLP-DBA in predicting bandwidth demand and improving uplink latency performance. The uplink latency is defined as the duration between the time a packet arrives at a buffer of an ONU-AP until its uplink transmission. Latency constraints, Dst = 1 and 10 ms, are considered in supporting H2M interactions [1], [2]. Several observations can be highlighted. First of all, compared to the limited-service DBA, MR-DBA, LSTP-DBA, and the proposed MLP-DBA without classification, the uplink latency performance of the MLP-DBA with classification is improved under light-to-moderate network loads, i.e., 0.1∼0.5 in Fig. 8(a) and 0.1∼0.4 in Fig. 8(b). This is attributed to the high accuracy of ON/OFF status prediction, BWpred estimation and the subsequent allocation of BWdem = BWpred + BWreq to ensure that arriving packets can be transmitted without having to wait for the next transmission cycle. MLP-DBA without classification achieve better performance compared to MR-DBA and LSTP-DBA due to its superior ON/OFF status prediction performance of the ANN as compared to the most-recent and the 4-order autoregressive models, respectively. However, MR-DBA, LSTP-DBA, and MLP-DBA without classification result in high uplink latency similar to that of the limited-service DBA algorithm when network load is beyond 0.5 in Fig. 8(a) and (b). This is because under heavy network loads, packet bursts at ONU-APs tend to have shorter OFF intervals, leading to Authorized licensed use limited to: Microchip Tech Inc. Downloaded on January 24,2022 at 13:12:05 UTC from IEEE Xplore. Restrictions apply. 3750 IEEE INTERNET OF THINGS JOURNAL, VOL. 6, NO. 2, APRIL 2019 (a) (b) Fig. 8. Average uplink latency with the proposed MLP-DBA. (a) 16 ONU-APs. (b) 32 ONU-APs. accumulated packet queues at ONU-APs. Consequently, conventional predictive DBAs that allocates min{BWdem , BWmax } to ONU-APs are less effective when BWmax is allocated in each polling cycle and uplink latency is mainly attributed to multiple-cycle waiting time before a packet is allocated bandwidth for transmission. Adaptively allocating bandwidth to ONU-APs based on the proposed classification addresses this latency bottleneck. As shown in Fig. 8(a) and (b), implementing MLP-DBA with ONU-AP classification effectively reduces the uplink latency even under heavy network loads. This is because bandwidth is allocated to heavily loaded Class C ONU-APs, by adding additional transmission opportunities in each polling cycle. In comparison, for network loads below 0.5 in the 16-ONU-AP network and 0.3 in the 32-ONU-AP network, MLP-DBA with and without classification exhibit similar latency performance since Class-A ONU-APs form the majority type of ONU-APs. Overall, considering H2M applications, MLP-DBA is shown to support approximately 10% more network traffic than conventional DBA algorithms under Dst = 1 and 10 ms. As presented in Fig. 8, with MLP-DBA, a 16-ONU-AP network can support a network traffic load of up to 0.6 and 0.8 under Dst = 1 ms and Dst = 10 ms, respectively, and a 32-ONU-AP network can support a network load of up to 0.4 for Dst = 1 ms and 0.6 for Dst = 10 ms. Fig. 9 plots the packet drop ratio under limited-service DBA, MR-DBA, LSTP-DBA, and MLP-DBA with/without Fig. 9. Distribution of Class A, B, and C ONU-APs. (a) 16 ONU-APs. (b) 32 ONU-APs. classification. As shown in the figures, implementations of MR-DBA, LSTP-DBA, and MLP-DBA without classification help alleviate packet drops under heavy network loads. However, the improvement shown is marginal with < 5% compared to limited-service DBA in both networks. For a 32ONU-AP network, the packet drop ratio is about twice that in a 16-ONU-AP network. This is attributed to the fact that under the same maximum polling cycle duration of 1 ms, the BWmax of a 32-ONU-AP network is half of that for a 16-ONUAP network. In comparison, the proposed MLP-DBA based on ONU-AP classification shows significant improvement in reducing packet drops. Overall, as illustrated in the figures, the packet drop ratio for MLP-DBA with classification is reduced by 10% as compared to the baseline limited-service DBA at high traffic loads. This is due to the reduced polling cycle durations for heavily loaded Class C ONU-APs by introducing additional transmission opportunities. Since MLP-DBA can predict and classify ONU-APs that have long packet queues, GATE messages are generated prior to receiving REPORT messages from these ONU-APs. As a result, the polling cycle duration of these ONU-APs is reduced, and additional bandwidth is allocated, thereby improving both latency and packet drop performances. The performance of the proposed MLP-DBA can be further explained by viewing the varying distribution of Class-A, B, and C ONU-APs in the two networks in Fig. 10. Authorized licensed use limited to: Microchip Tech Inc. Downloaded on January 24,2022 at 13:12:05 UTC from IEEE Xplore. Restrictions apply. RUAN et al.: MACHINE LEARNING-BASED BANDWIDTH PREDICTION FOR LOW-LATENCY H2M APPLICATIONS 3751 in the 32-ONU-AP network, when network load increases, more ONU-APs are classified as Class C in the 32-ONU-AP network. Overall, results in Fig. 10 verify the capability of the proposed MLP-DBA in making adaptive bandwidth allocation decisions that reduces uplink latency and packet drops. Concluding, simulations presented in Figs. 8–10 in this section highlight the importance of classification and adaptive bandwidth allocation decisions. VI. C ONCLUSION (a) (b) Fig. 10. Distribution of Class A, B, and C ONU-APs. (a) 16 ONU-APs. (b) 32 ONU-APs. To improve the uplink latency performance for H2M traffic over access networks, we proposed MLP-DBA to adaptively allocate bandwidth to classified ONU-APs based on ANN-enabled bursty ON/OFF status prediction and bandwidth estimation. Specifically, we first presented our exploitation of an ANN at the CO to predict the ON/OFF status of ONU-APs in each polling cycle. We showed that with supervised training, the proposed ANN achieved >90% accuracy in identifying the ON/OFF status of ONU-APs, thereby yielding superior ONU-AP bandwidth estimation as compared to MRand LSTP-DBA algorithms. Further, based on both ON/OFF prediction and the estimated bandwidth, the proposed MLPDBA was able to: 1) allocate the estimated bandwidth to ONU-APs that are in ON intervals (Class A); 2) reserve bandwidth from ONU-APs that are in OFF intervals (Class B); and 3) concurrently providing additional transmission opportunities to heavily-loaded ONU-APs (Class C). Consequently, MLP-DBA effectively reduced the uplink latency and alleviated packet drops as compared to the baseline limited-service DBA and predictive MR-DBA and LSTP-DBA algorithms. B. ONU-AP Classification Analysis In each polling cycle, the CO classifies ONU-APs upon receiving the REPORTs. Fig. 10(a) and (b) show the distribution of the classification results of ONU-APs, i.e., Class A, B, and C, during MLP-DBA operation over 500 s simulation time in 16-ONU-AP and 32-ONU-AP network, respectively. In the lightly-loaded region, i.e., for normalized network loads under 0.4, Class-B ONU-APs dominate since the ONU-APs tend to experience long OFF intervals. In this case, when an ONU-AP occasionally encounters an ON interval, arriving packets can be transmitted within BWmax . When the network traffic load increases, the percentage of Class A increases and followed by Class C, which rapidly increases and dominates the network due to the heavy packet bursts at these ONU-APs. In this case, conventional predictive DBAs become less effective as BWmax is allocated in majority of the polling cycles due to the dominated Class-C ONU-APs. Therefore, marginal improvement in reducing latency and packet drop ratio is shown in Figs. 8 and 9 when network loads are beyond 0.5 with MRDBA, LSTP-DBA, and MLP-DBA without classification. In contrast, MLP-DBA with the proposed ONU-AP classification adaptively allocates BWdem to Class-A ONU-APs and enables additional transmission opportunities to heavily loaded Class-C ONU-APs, thereby effectively reducing latency and packet drop ratio as presented in Figs. 8 and 9. Moreover, since bandwidth contention amongst ONU-APs is more competitive R EFERENCES [1] G. P. Fettweis, “The Tactile Internet: Applications and challenges,” IEEE Veh. Technol. Mag., vol. 9, no. 1, pp. 64–70, Mar. 2014. [2] M. Simsek, A. Aijaz, M. Dohler, J. Sachs, and G. Fettweis, “5Genabled Tactile Internet,” IEEE J. Sel. Areas Commun., vol. 34, no. 3, pp. 460–473, Mar. 2016. [3] M. Dohler et al., “Internet of skills: Where robotics meets AI, 5G and the Tactile Internet,” in Proc. EuCNC, Oulu, Finland, Jun. 2017, pp. 1–5. [4] E. Wong, M. P. I. Dias, and L. Ruan, “Predictive resource allocation for Tactile Internet capable passive optical LANs,” J. Lightw. Technol., vol. 35, no. 13, pp. 2629–2641, Jul. 1, 2017. [5] M. Chowdhury and M. Maier, “Collaborative computing for advanced Tactile Internet human-to-robot (H2R) communications in integrated FiWi multirobot infrastructures,” IEEE Internet Things J., vol. 4, no. 6, pp. 2142–2158, Dec. 2017. [6] M. Chowdhury, E. Steinbach, W. Kellerer, and M. Maier, “Context-aware task migration for HART-centric collaboration over FiWi based Tactile Internet infrastructures,” IEEE Trans. Parallel Distrib. Syst., vol. 29, no. 6, pp. 1231–1246, Jun. 2018. [7] E. Wong, M. P. I. Dias, and L. Ruan, “Tactile Internet capable passive optical LAN for healthcare,” in Proc. OECC, Niigata, Japan, Jul. 2016, pp. 1–3. [8] S. Mondal, G. Das, and E. Wong, “A novel cost optimization framework for multi-cloudlet environment over optical access networks,” in Proc. IEEE GLOBECOM, Singapore, Dec. 2017, pp. 1–7. [9] M. Condoluci et al., “Soft resource reservation for low-latencyed teleoperation over mobile networks,” IEEE Access, vol. 5, pp. 10445–10455, 2017. [10] X. Xu, Q. Liu, and E. Steinbach, “Toward QoE-driven dynamic control scheme switching for time-latencyed teleoperation systems: A dedicated case study,” in Proc. IEEE HAVE, Oct. 2017, pp. 1–6. Authorized licensed use limited to: Microchip Tech Inc. Downloaded on January 24,2022 at 13:12:05 UTC from IEEE Xplore. Restrictions apply. 3752 IEEE INTERNET OF THINGS JOURNAL, VOL. 6, NO. 2, APRIL 2019 [11] M. Condoluci, G. Araniti, T. Mahmoodi, and M. Dohler, “Enabling the IoT machine age with 5G: Machine-type multicast services for innovative real-time applications,” IEEE Access, vol. 4, pp. 5555–5569, 2016. [12] G. Kramer, Ethernet Passive Optical Networks. New York, NY, USA: McGraw-Hill, 2005. [13] Y. Luo and N. Ansari, “Bandwidth allocation for multiservice access on EPONs,” IEEE Commun. Mag., vol. 43, no. 2, pp. 16–21, Feb. 2005. [14] M. P. Mcgarry, M. Reisslein, and M. Maier, “Ethernet passive optical network architectures and dynamic bandwidth allocation algorithms,” IEEE Commun. Surveys Tuts., vol. 10, no. 3, pp. 46–60, 3rd Quart., 2008. [15] I. Mamounakis, K. Yiannopoulos, G. Papadimitriou, and E. Varvarigos, “On the prediction of EPON traffic using polynomial fitting in optical network units,” in Proc. OPTICS, Vienna, Austria, Aug. 2014, pp. 1–7. [16] R. Kubo, J. I. Kani, Y. Fujimoto, N. Yoshimoto, and K. Kumozaki, “Adaptive power saving mechanism for 10 gigabit class PON systems,” IEICE Trans. Commun., vol. E93–B, no. 2, pp. 280–288, 2010. [17] M. Fiammecngo, A. Lindström, P. Monti, L. Wosinska, and B. Skubic, “Experimental evaluation of cyclic sleep with adaptable sleep period length for PON,” in Proc. 37th Eur. Conf. Exhibit. Opt. Commun., Geneva, Switzerland, Sep. 2011, pp. 1–3. [18] M. P. I. Dias, B. S. Karunaratne, and E. Wong, “Bayesian estimation and prediction-based dynamic bandwidth allocation algorithm for sleep/doze-mode passive optical networks,” J. Lightw. Technol., vol. 32, no. 14, pp. 2560–2568, Jul. 15, 2014. [19] H.-J. Byun, J. -M. Nho, and J.-T. Lim, “Dynamic bandwidth allocation algorithm in Ethernet passive optical networks,” Elect. Lett., vol. 39, no. 13, pp. 1001–1002, Jun. 2003. [20] Y. Luo and N. Ansari, “Limited sharing with traffic prediction for dynamic bandwidth allocation and QoS provisioning over EPONs,” J. Opt. Netw., vol. 4, no. 9, pp. 561–572, Sep. 2005, . [21] P. Sarigiannidis, D. Pliatsios, T. Zygiridis, and N. Kantartzis, “DAMA: A data mining forecasting DBA scheme for XG-PONs,” in Proc. MOCAST, Thessaloniki, Greece, May 2016, pp. 1–4. [22] L. Ruan, S. Mondal, and E. Wong, “Machine learning based bandwidth prediction in Tactile heterogeneous access networks,” in Proc. INFOCOM (WORKSHOP), Honolulu, HI, USA, Apr. 2018, pp. 1–2. [23] N. Kato et al., “The deep learning vision for heterogeneous network traffic control: proposal, challenges, and future perspective,” IEEE Wireless Commun., vol. 24, no. 3, pp. 146–153, Jun. 2017. [24] E. Wong, “Current and next-generation broadband access network technologies,” J. Lightw. Technol., vol. 30, no. 4, pp. 597–608, 2012. [25] S. Haykin, Neural Networks: A Comprehensive Foundation, 2nd ed. New Delhi, India: Prentice-Hall, 1999. [26] W. H. Tranter et al., “On the self-similar nature of Ethernet traffic (extended version),” in The Best of the Best: Fifty Years of Communications and Networking Research. Piscataway, NJ, USA: IEEE Press, 2007, pp. 517–531. [27] R. Fontugne et al., “Scaling in Internet traffic: A 14 year and 3 day longitudinal study, with multiscale analyses and random projections,” IEEE/ACM Trans. Netw., vol. 25, no. 4, pp. 2152–2165, Aug. 2017. [28] P. Loiseau et al., “Investigating self-similarity and heavy-tailed distributions on a large-scale experimental facility,” IEEE/ACM Trans. Netw., vol. 18, no. 4, pp. 1261–1274, Aug. 2010. Lihua Ruan (GS’16) received the M.S. degree from Northwestern Polytechnical University, Xi’an, China, in 2015. She is currently pursuing the Ph.D. degree at the University of Melbourne, Melbourne, VIC, Australia. Her current research interests include smart body area networks, wireless local area networks, and dynamic bandwidth resource allocation in passive optical networks, in conjunction with the use of machine learning techniques, for tactile Internet applications. Maluge Pubuduni Imali Dias received the B.Sc. degree (Hons.) from the University of Moratuwa, Moratuwa, Sri Lanka, in 2007, and the Ph.D. degree from the University of Melbourne, Melbourne, VIC, Australia, in 2016. She is currently engaged in teaching and research activities with the University of Melbourne. Her current research interests include dynamic bandwidth allocation algorithms, energy-efficiency, tactile Internet, and intelligent transportation systems. Elaine Wong (SM’14) received the B.E. and Ph.D. degrees from the University of Melbourne, Melbourne, VIC, Australia. She is currently a Professor and an Australia Research Council Future Fellow with the University of Melbourne. She has coauthored over 150 journal and conference publications. Her current research interests include energyefficient optical and wireless networks, optical-wireless integration, broadband applications of vertical-cavity surface-emitting lasers, wireless sensor body area networks, and emerging optical and wireless technologies for tactile Internet. Dr. Wong has served on the Editorial Board of the Journal of Lightwave Technology and the Journal of Optical Communications and Networking. Authorized licensed use limited to: Microchip Tech Inc. Downloaded on January 24,2022 at 13:12:05 UTC from IEEE Xplore. Restrictions apply.