Context and Motivation

Artificial intelligence (AI) is becoming a pervasive technology in ICT systems development, as well as for its presence in products and services of daily use. The pervasive use of neural-based methodologies and technologies comes at the cost of increased functional and non-functional requirements related to fundamental properties, such as energy efficiency, ability to continuously adapt to changing conditions and tasks, dependability, security and human-machine interactivity. While compelling, such requirements can also be seen as research opportunities to design and develop novel AI methodologies that are efficient and effective in meeting such requirements, e.g., developing continual learning strategies for lifelong adaptation or randomised ant tiny neural networks to facilitate learning on constrained devices. Pervasiveness has another interpretation linked to the need to develop computing and communication technologies specifically meant to support the deployment and execution of AI models, e.g., neuromorphic hardware or high-performance computing for AI acceleration. The event we propose focuses on the novel and thriving research field of Pervasive AI, lying at the crossroads of artificial intelligence and pervasive computing.

---

Tutorial on Continual learning in distributed and heterogeneous systems

This tutorial will cover fundamental and state-of-the-art AI methodologies and models enabling the design of efficient and effective distributed and embedded neural learning systems, including reservoir computing, federated learning and continual learning. The view on AI methods will be complemented by an introduction to relevant pervasive computing and communication abstractions, infrastructures, and applications for AI. Relevant programming libraries and tools will also be introduced whenever appropriate. The tutorial targets early-career researchers seeking ideas for their doctoral studies and more advanced-stage researchers looking to enter a novel AI field, rapidly building momentum and seeking foundational knowledge and a perspective on current research.

The ideal audience would have a foundational knowledge of learning-based methodology without necessarily being experienced in AI research. We want to be inclusive of a broader audience of researchers with different backgrounds and interests, including Artificial Intelligence, Cloud/Edge/IoT, Deep Learning, HPC, Machine Learning and Pervasive computing.

The benefit to participants will be two-fold. On the one hand, they will be introduced to knowledge about a toolset of models and methodologies relevant to the Pervasive AI topic but which also have value and impact per-se (e.g., continual learning, neuromorphic computing, federated learning, reservoir computing, etc.). On the other hand, the participants will also benefit from the opportunity to be early members of a novel and growing research community. We believe the topic of the tutorial to be extremely timely, given the worldwide interest towards some of its key topics, such as continual learning (arguably the fastest-growing theme in the neural network conferences in 2021-23), sustainable AI (which is a major theme in the top generalist AI venues), federated learning. In addition to this, several national and international initiatives (e.g., DARPA, Horizon Europe, etc.) are vigorously promoting research which deals with fundamental topics in Pervasive AI, including learning at the edge and edge AI, neuromorphic computing, and general sustainable AI, for which heavy research financing is foreseen in the coming years.

Topics

The tutorial is organised into four modules (each of about 45 minutes). The first module will introduce the basic concepts of efficient and scalable neural network design for pervasive computing. The second module will introduce and delve into the solutions, architectures and approaches enabling distributed, pervasive and federated learning, with a specific focus on this latter, both from a computational and communication standpoint. The third module will build on the foundational concepts introduced in previous modules to discuss the most recent advances in neural networks in the cloud-edge continuum, spanning from scalable and federated deep reservoir computing to efficient neuromorphic architectures. The fourth and last module will conclude the tutorial with an introduction to continual learning and its most recent research challenges associated with learning in pervasive computing settings.

Slides

• Part 1 – Fundamentals of Pervasive AI
• Part 2 – Solutions and Infrastructures for distributed and federated learning
• Part 3 – Advanced Models
• Part 4 – Continual Learning

Tutorial Instructors