Nicola Di Cicco

2023

1. IEEE TNSM

   DeepLS: Local Search for Network Optimization based on Lightweight Deep Reinforcement Learning

   Nicola Di Cicco, Memedhe Ibrahimi, and 2 more authors

   IEEE Transactions on Network and Service Management, 2023

   Abs Bib PDF Code

   Deep Reinforcement Learning (DRL) is being investigated as a competitive alternative to traditional techniques for solving network optimization problems. A promising research direction lies in enhancing traditional optimization algorithms by offloading low-level decisions to a DRL agent. In this study, we consider how to effectively employ DRL to improve the performance of Local Search algorithms, i.e., algorithms that, starting from a candidate solution, explore the solution space by iteratively applying local changes (i.e., moves), yielding the best solution found in the process. We propose a Local Search algorithm based on lightweight Deep Reinforcement Learning (DeepLS) that, given a neighborhood, queries a DRL agent for choosing a move, with the goal of achieving the best objective value in the long term. Our DRL agent, based on permutation-equivariant neural networks, is composed by less than a hundred parameters, requiring only up to ten minutes of training and can evaluate problem instances of arbitrary size, generalizing to networks and traffic distributions unseen during training. We evaluate DeepLS on two illustrative NP-Hard network routing problems, namely OSPF Weight Setting and Routing and Wavelength Assignment, training on a single small network only and evaluating on instances 2x-10x larger than training. Experimental results show that DeepLS outperforms existing DRL-based approaches from literature and attains competitive results with state-of-the-art metaheuristics, with computing times up to 8x smaller than the strongest algorithmic baselines.

   @article{dicicco2023deepls,
     author = {Di Cicco, Nicola and Ibrahimi, Memedhe and Troia, Sebastian and Tornatore, Massimo},
     journal = {IEEE Transactions on Network and Service Management},
     title = {DeepLS: Local Search for Network Optimization based on Lightweight Deep Reinforcement Learning},
     year = {2023},
     pages = {1-1},
     doi = {10.1109/TNSM.2023.3287433}
   }

2. IEEE NetSoft

   DRL-FORCH: A Scalable Deep Reinforcement Learning-based Fog Computing Orchestrator

   Nicola Di Cicco, Gaetano Francesco Pittalà, and 5 more authors

   In 2023 IEEE 9th International Conference on Network Softwarization (NetSoft), 2023

   Abs Bib PDF Code

   We consider the problem of designing and training a neural network-based orchestrator for fog computing service deployment. Our goal is to train an orchestrator able to optimize diversified and competing QoS requirements, such as blocking probability and service delay, while potentially supporting thousands of fog nodes. To cope with said challenges, we implement our neural orchestrator as a Deep Set (DS) network operating on sets of fog nodes, and we leverage Deep Reinforcement Learning (DRL) with invalid action masking to find an optimal trade-off between competing objectives. Illustrative numerical results show that our Deep Set-based policy generalizes well to problem sizes (i.e., in terms of numbers of fog nodes) up to two orders of magnitude larger than the ones seen during the training phase, outperforming both greedy heuristics and traditional Multi-Layer Perceptron (MLP)-based DRL. In addition, inference times of our DS-based policy are up to an order of magnitude faster than an MLP, allowing for excellent scalability and near real-time online decision-making.

   @inproceedings{dicicco2023netsoft,
     author = {Cicco, Nicola Di and Pittalà, Gaetano Francesco and Davoli, Gianluca and Borsatti, Davide and Cerroni, Walter and Raffaelli, Carla and Tornatore, Massimo},
     booktitle = {2023 IEEE 9th International Conference on Network Softwarization (NetSoft)},
     title = {DRL-FORCH: A Scalable Deep Reinforcement Learning-based Fog Computing Orchestrator},
     year = {2023},
     pages = {125-133},
     doi = {10.1109/NetSoft57336.2023.10175398},
   }

2022

1. IEEE JSTQE

   On Deep Reinforcement Learning for Static Routing and Wavelength Assignment

   Nicola Di Cicco, Emre Furkan Mercan, and 5 more authors

   IEEE Journal of Selected Topics in Quantum Electronics, 2022

   Abs Bib PDF

   Deep Reinforcement Learning (DRL) is rising as a promising tool for solving optimization problems in optical networks. Though studies employing DRL for solving static optimization problems in optical networks are appearing, assessing strengths and weaknesses of DRL with respect to state-of-the-art solution methods is still an open research question. In this work, we focus on Routing and Wavelength Assignment (RWA), a well-studied problem for which fast and scalable algorithms leading to better optimality gaps are always sought for. We develop two different DRL-based methods to assess the impact of different design choices on DRL performance. In addition, we propose a Multi-Start approach that can improve the average DRL performance, and we engineer a shaped reward that allows efficient learning in networks with high link capacities. With Multi-Start, DRL gets competitive results with respect to a state-of-the-art Genetic Algorithm with significant savings in computational times. Moreover, we assess the generalization capabilities of DRL to traffic matrices unseen during training, in terms of total connection requests and traffic distribution, showing that DRL can generalize on small to moderate deviations with respect to the training traffic matrices. Finally, we assess DRL scalability with respect to topology size and link capacity.

   @article{dicicco2022jstqe,
     author = {Di Cicco, Nicola and Mercan, Emre Furkan and Karandin, Oleg and Ayoub, Omran and Troia, Sebastian and Musumeci, Francesco and Tornatore, Massimo},
     journal = {IEEE Journal of Selected Topics in Quantum Electronics},
     title = {On Deep Reinforcement Learning for Static Routing and Wavelength Assignment},
     year = {2022},
     volume = {28},
     number = {4: Mach. Learn. in Photon. Commun. and Meas. Syst.},
     pages = {1-12},
     doi = {10.1109/JSTQE.2022.3151323},
   }