2023
Conference article
Open Access
Supporting users in controlling automations in intelligent environments through conversational agents
Gallo S
The integration of sensors and intelligent devices in everyday environments is changing the way people interact with common objects. The need to provide non-expert users with an easy and efficient way to customize the behaviour of these devices according to their preferences and habits is crucial. T... his paper presents the work carried out by the author in the development of a conversational agent to empower the end-user in controlling intelligent environments, allowing for easy and fast communication with sensors, actuators, and smart objects using trigger-action customisation rules. The paper provides an overview of the state of the art in research and commercial solutions for conversational agents and presents some preliminary results of the research. Then, the ongoing work on a Systematic Literature Review and the migration to open-source solutions are presented. Finally, an overview of future work is given. [show more]Source: CEUR WORKSHOP PROCEEDINGS. Cagliari, Italy, 06/06/2023
Gallo S
The integration of sensors and intelligent devices in everyday environments is changing the way people interact with common objects. The need to provide non-expert users with an easy and efficient way to customize the behaviour of these devices according to their preferences and habits is crucial. T... his paper presents the work carried out by the author in the development of a conversational agent to empower the end-user in controlling intelligent environments, allowing for easy and fast communication with sensors, actuators, and smart objects using trigger-action customisation rules. The paper provides an overview of the state of the art in research and commercial solutions for conversational agents and presents some preliminary results of the research. Then, the ongoing work on a Systematic Literature Review and the migration to open-source solutions are presented. Finally, an overview of future work is given. [show more]Source: CEUR WORKSHOP PROCEEDINGS. Cagliari, Italy, 06/06/2023
See at:
ceur-ws.org 
| CNR IRIS | ISTI Repository | CNR IRIS 
2023
Conference article
Open Access
Galliz at GeoLingIt: enhancing BERT with vocabulary knowledge for predicting the region of language varieties of Italy
Labruna T, Gallo S
The linguistic diversity of the Italian peninsula and its islands, characterized by several language varieties, represents a linguistic condition and a cultural treasure unique in Europe. However, the oral nature of these varieties poses a challenge to their preservation in the written form. While s... ignificant research efforts have been dedicated to standard Italian language processing, less attention has been given to the language varieties of Italy and the development of supporting resources. This paper aims to study the peculiarities of language varieties of Italy and identify the region of origin of tweets written in non-[Standard Italian] varieties. To achieve this goal, we utilized two main techniques: fine-tuning a language model (BERT) and implementing an algorithm that utilizes dictionaries of regional varieties and word frequency. Our results show that integrating lexical analysis with BERT could be a promising approach for this particular task. We present an overview of the data, methodology, and evaluation results, then discuss the implications of our findings. [show more]Source: CEUR WORKSHOP PROCEEDINGS. Parma, Italy, 07-08/08/2023
Labruna T, Gallo S
The linguistic diversity of the Italian peninsula and its islands, characterized by several language varieties, represents a linguistic condition and a cultural treasure unique in Europe. However, the oral nature of these varieties poses a challenge to their preservation in the written form. While s... ignificant research efforts have been dedicated to standard Italian language processing, less attention has been given to the language varieties of Italy and the development of supporting resources. This paper aims to study the peculiarities of language varieties of Italy and identify the region of origin of tweets written in non-[Standard Italian] varieties. To achieve this goal, we utilized two main techniques: fine-tuning a language model (BERT) and implementing an algorithm that utilizes dictionaries of regional varieties and word frequency. Our results show that integrating lexical analysis with BERT could be a promising approach for this particular task. We present an overview of the data, methodology, and evaluation results, then discuss the implications of our findings. [show more]Source: CEUR WORKSHOP PROCEEDINGS. Parma, Italy, 07-08/08/2023
See at:
ceur-ws.org | CNR IRIS | ISTI Repository | CNR IRIS
2022
Conference article
Open Access
A conversational agent for creating flexible daily automation
Gallo S., Paternò F.
The spread of sensors and intelligent devices of the Internet of things and their integration in daily environments are changing the way we interact with some of the most common objects in everyday life. Therefore, there is an evident need to provide non-expert users with the ability to customize in... a simple but effective way the behaviour of these devices based on their preferences and habits. This paper presents RuleBot, a conversational agent that uses machine learning and natural language processing techniques to allow end users to create automation according to a flexible implementation of the trigger-action paradigm, and thereby customize the behaviour of devices and sensors using natural language. In particular, the paper describes the design and implementation ofRuleBot, and reports on a user test and lessons learnt [show more]DOI: 10.1145/3531073.3531090
Metrics:
Gallo S., Paternò F.
The spread of sensors and intelligent devices of the Internet of things and their integration in daily environments are changing the way we interact with some of the most common objects in everyday life. Therefore, there is an evident need to provide non-expert users with the ability to customize in... a simple but effective way the behaviour of these devices based on their preferences and habits. This paper presents RuleBot, a conversational agent that uses machine learning and natural language processing techniques to allow end users to create automation according to a flexible implementation of the trigger-action paradigm, and thereby customize the behaviour of devices and sensors using natural language. In particular, the paper describes the design and implementation ofRuleBot, and reports on a user test and lessons learnt [show more]DOI: 10.1145/3531073.3531090
Metrics:
See at:
dl.acm.org | CNR IRIS | ISTI Repository | CNR IRIS | CNR IRIS
2023
Conference article
Open Access
Towards a chatbot for creating trigger-action rules based on ChatGPT and Rasa
Gallo S, Malizia A, Paternò F
In this paper, we propose a novel approach for building a conversational agent for creating trigger-action rules and controlling smart objects inside smart environments, such as a smart home. Our approach integrates ChatGPT, a state-of-the-art pre-trained language model for open-domain dialogue ge... neration, with Rasa, a popular open-source framework for developing task-oriented chatbots. We leverage ChatGPT's abilities to perform Natural Language Processing tasks through prompting and few-shot learning, and Rasa Open Source's features to handle intents, entities, forms, and execute actions. We design Rasa custom actions that invoke ChatGPT's API to process complex customization rules, manage conversational breakdowns and answer questions about the smart environment. [show more]Source: CEUR WORKSHOP PROCEEDINGS. Cagliari, Italy, 06/06/2023
Gallo S, Malizia A, Paternò F
In this paper, we propose a novel approach for building a conversational agent for creating trigger-action rules and controlling smart objects inside smart environments, such as a smart home. Our approach integrates ChatGPT, a state-of-the-art pre-trained language model for open-domain dialogue ge... neration, with Rasa, a popular open-source framework for developing task-oriented chatbots. We leverage ChatGPT's abilities to perform Natural Language Processing tasks through prompting and few-shot learning, and Rasa Open Source's features to handle intents, entities, forms, and execute actions. We design Rasa custom actions that invoke ChatGPT's API to process complex customization rules, manage conversational breakdowns and answer questions about the smart environment. [show more]Source: CEUR WORKSHOP PROCEEDINGS. Cagliari, Italy, 06/06/2023
See at:
ceur-ws.org | CNR IRIS | ISTI Repository | CNR IRIS
2023
Conference article
Open Access
Conversational interfaces in IoT ecosystems: where we are, what is still missing
Gallo S, Paternò F, Malizia A
In the last few years, text and voice-based conversational agents have become more and more popular all over the world as virtual assistants for a variety of tasks. In addition, the deployment on the market of many smart objects connected with these agents has introduced the possibility of controlli... ng and personalising the behaviour of several connected objects using natural language. This has the potential to allow people, also those without a technical background, to effectively control and use the wide variety of connected objects and services. In this paper, we present an analysis of how conversational agents have been used to interact with smart environments (such as smart homes). For this purpose, we have carried out a systematic literature review considering publications selected from the ACM and IEEE digital libraries to investigate the technologies used to design and develop conversational agents for IoT settings, including Artificial Intelligence techniques, the purpose that they have been used for, and the level of user involvement in such studies. The resulting analysis is useful to better understand how this field is evolving and indicate the challenges still open in this area that should be addressed in future research work to allow people to completely benefit from this type of solution. [show more]DOI: 10.1145/3626705.3627775
Metrics:
Gallo S, Paternò F, Malizia A
In the last few years, text and voice-based conversational agents have become more and more popular all over the world as virtual assistants for a variety of tasks. In addition, the deployment on the market of many smart objects connected with these agents has introduced the possibility of controlli... ng and personalising the behaviour of several connected objects using natural language. This has the potential to allow people, also those without a technical background, to effectively control and use the wide variety of connected objects and services. In this paper, we present an analysis of how conversational agents have been used to interact with smart environments (such as smart homes). For this purpose, we have carried out a systematic literature review considering publications selected from the ACM and IEEE digital libraries to investigate the technologies used to design and develop conversational agents for IoT settings, including Artificial Intelligence techniques, the purpose that they have been used for, and the level of user involvement in such studies. The resulting analysis is useful to better understand how this field is evolving and indicate the challenges still open in this area that should be addressed in future research work to allow people to completely benefit from this type of solution. [show more]DOI: 10.1145/3626705.3627775
Metrics:
See at:
dl.acm.org | CNR IRIS | ISTI Repository | CNR IRIS
2024
Conference article
Open Access
End-user personalisation of humanoid robot behaviour through vocal interaction
Gallo S., Vaiani G., Paternò F.
This study explores the integration of Large Language Models with social robots to facilitate End-User Development through natural language interactions. The paper presents a prototype system embodied in a Pepper robot that allows non-expert users to customise robot behaviours by defining personalis... ation rules via vocal commands. This system employs trigger-action programming, enabling users to create automations based on specific triggers and actions without requiring in-depth technical knowledge. Through an example scenario, we show how users can program the robot by employing voice commands to execute actions when an event occurs. The created automations can also involve available IoT objects. The study investigates the potential of natural language interaction to improve the usability and flexibility of robot programming, offering new possibilities for personalised interactions in various settings. [show more]Source: CEUR WORKSHOP PROCEEDINGS, vol. 3794. Arenzano, Italy, 3/06/2024
Gallo S., Vaiani G., Paternò F.
This study explores the integration of Large Language Models with social robots to facilitate End-User Development through natural language interactions. The paper presents a prototype system embodied in a Pepper robot that allows non-expert users to customise robot behaviours by defining personalis... ation rules via vocal commands. This system employs trigger-action programming, enabling users to create automations based on specific triggers and actions without requiring in-depth technical knowledge. Through an example scenario, we show how users can program the robot by employing voice commands to execute actions when an event occurs. The created automations can also involve available IoT objects. The study investigates the potential of natural language interaction to improve the usability and flexibility of robot programming, offering new possibilities for personalised interactions in various settings. [show more]Source: CEUR WORKSHOP PROCEEDINGS, vol. 3794. Arenzano, Italy, 3/06/2024
See at:
ceur-ws.org | CNR IRIS | CNR IRIS
2024
Journal article
Open Access
A conversational agent for creating automations exploiting large language models
Gallo S., Paternò F., Malizia A.
The proliferation of sensors and smart Internet of Things (IoT) devices in our everyday environments is reshaping our interactions with everyday objects. This change underlines the need to empower non-expert users to easily configure the behaviour of these devices to align with their preferences and... habits. At the same time, recent advances in generative transformers, such as ChatGPT, have opened up new possibilities in a variety of natural language processing tasks, enhancing reasoning capabilities and conversational interactions. This paper presents RuleBot + + , a conversational agent that exploits GPT-4 to assist the user in the creation and modification of trigger-action automations through natural language. After an introduction to motivations and related work, we present the design and implementation of RuleBot + + and report the results of the user test in which users interacted with our solution and Home Assistant, one of the most used open-source tools for managing smart environments. [show more]Source: PERSONAL AND UBIQUITOUS COMPUTING
DOI: 10.1007/s00779-024-01825-5
Metrics:
Gallo S., Paternò F., Malizia A.
The proliferation of sensors and smart Internet of Things (IoT) devices in our everyday environments is reshaping our interactions with everyday objects. This change underlines the need to empower non-expert users to easily configure the behaviour of these devices to align with their preferences and... habits. At the same time, recent advances in generative transformers, such as ChatGPT, have opened up new possibilities in a variety of natural language processing tasks, enhancing reasoning capabilities and conversational interactions. This paper presents RuleBot + + , a conversational agent that exploits GPT-4 to assist the user in the creation and modification of trigger-action automations through natural language. After an introduction to motivations and related work, we present the design and implementation of RuleBot + + and report the results of the user test in which users interacted with our solution and Home Assistant, one of the most used open-source tools for managing smart environments. [show more]Source: PERSONAL AND UBIQUITOUS COMPUTING
DOI: 10.1007/s00779-024-01825-5
Metrics:
See at:
Personal and Ubiquitous Computing | IRIS Cnr | IRIS Cnr | Archivio della Ricerca - Università di Pisa | Archivio della Ricerca - Università di Pisa | CNR IRIS
2024
Journal article
Open Access
Human control in green smart homes
Gallo S., Maenza S., Mattioli A., Paternò F.
We present an approach to supporting human control in green smart homes. The approach is based on a set of tools for allowing end users to create automations through innovative interaction modalities, execute them, and re- ceive relevant explanations about the resulting behaviours and their relevanc... e for the user goals, in particular those that are connected to sustainability principles. [show more]Source: ERCIM NEWS, vol. 138, pp. 28-29
Project(s): End-User Development of Automations for Explainable Green Smart Homes
Gallo S., Maenza S., Mattioli A., Paternò F.
We present an approach to supporting human control in green smart homes. The approach is based on a set of tools for allowing end users to create automations through innovative interaction modalities, execute them, and re- ceive relevant explanations about the resulting behaviours and their relevanc... e for the user goals, in particular those that are connected to sustainability principles. [show more]Source: ERCIM NEWS, vol. 138, pp. 28-29
Project(s): End-User Development of Automations for Explainable Green Smart Homes
See at:
ercim-news.ercim.eu | CNR IRIS | CNR IRIS
2024
Conference article
Open Access
What people think about green smart homes
Barricelli B. R., Fogli D., Gallo S., Guizzardi D., Maenza S., Mattioli A., Paternò F.
Current technological trends have enabled several smart spaces in our daily environments. A typical example is the home, which is becoming more and more populated by connected objects and sensors. In this way, even people who are not professional software developers can create automations that compo... se the dynamic behaviours of such connected objects. In this paper, we report and discuss two online surveys targeting two different groups of users: one consists of a general population of end users with different degrees of experience in smart home use and personalization, and the other composed of people who have already some experiences in directly creating automations and managing smart environments. Overall, such studies provide useful information for better understanding the requirements that end-user development approaches should address to be successful in order to be actually adopted in such contexts. [show more]Source: CEUR WORKSHOP PROCEEDINGS, vol. 3685. Arenzano, Italy, 4/06/2024
Project(s): End-User Development of Automations for Explainable Green Smart Homes
Barricelli B. R., Fogli D., Gallo S., Guizzardi D., Maenza S., Mattioli A., Paternò F.
Current technological trends have enabled several smart spaces in our daily environments. A typical example is the home, which is becoming more and more populated by connected objects and sensors. In this way, even people who are not professional software developers can create automations that compo... se the dynamic behaviours of such connected objects. In this paper, we report and discuss two online surveys targeting two different groups of users: one consists of a general population of end users with different degrees of experience in smart home use and personalization, and the other composed of people who have already some experiences in directly creating automations and managing smart environments. Overall, such studies provide useful information for better understanding the requirements that end-user development approaches should address to be successful in order to be actually adopted in such contexts. [show more]Source: CEUR WORKSHOP PROCEEDINGS, vol. 3685. Arenzano, Italy, 4/06/2024
Project(s): End-User Development of Automations for Explainable Green Smart Homes
See at:
ceur-ws.org | CNR IRIS | CNR IRIS
2024
Conference article
Open Access
An architecture for green smart homes controlled by end users
Gallo S., Mattioli A., Paternò F., Barricelli B. R., Fogli D., Guizzardi D.
Automations can help support the achievement of various types of goals in a smart home. We present an architecture enabling the possibility of supporting user goals consistent with the principles of a circular economy (such as energy saving and waste recycling). It includes meta-design tools for cre... ating automations that better fit user needs and controlling their execution through innovative modalities based on conversational agents and augmented reality, and a home Digital Twin useful for supporting relevant simulations and analytics to understand the potential impact of specific automations on aspects relevant for circular economy. [show more]DOI: 10.1145/3656650.3656710
Project(s): End-User Development of Automations for Explainable Green Smart Homes
Metrics:
Gallo S., Mattioli A., Paternò F., Barricelli B. R., Fogli D., Guizzardi D.
Automations can help support the achievement of various types of goals in a smart home. We present an architecture enabling the possibility of supporting user goals consistent with the principles of a circular economy (such as energy saving and waste recycling). It includes meta-design tools for cre... ating automations that better fit user needs and controlling their execution through innovative modalities based on conversational agents and augmented reality, and a home Digital Twin useful for supporting relevant simulations and analytics to understand the potential impact of specific automations on aspects relevant for circular economy. [show more]DOI: 10.1145/3656650.3656710
Project(s): End-User Development of Automations for Explainable Green Smart Homes
Metrics:
See at:
IRIS Cnr | doi.org | Archivio istituzionale della ricerca - Università di Brescia | IRIS Cnr | doi.org | CNR IRIS | CNR IRIS
2021
Conference article
Open Access
Comparative analysis of composition paradigms for personalization rules in IoT settings
Gallo S., Manca M., Mattioli A., Paternò F., Santoro C.
The rapid pervasive diffusion of Internet of Things technologies has opened up many opportunities for people to directly personalise the behaviour of surrounding objects and devices based on the dynamic events that can occur. To this end, several tailoring environments have been proposed supporting ... the enduser creation of trigger-action rules. Such tools can support different composition paradigms. In this paper we present a study that analyses three composition paradigms (graphical wizard, block-based, and conversational) to better understand how well they support rule creation activities. In order to make the analysis consistent we considered three implementations of such composition paradigms supporting the same set of triggers and actions. We have carried out a first user study in order to gather empirical feedback for substantiating our analysis, which provides indications of the pros and cons of each approach. [show more]DOI: 10.1007/978-3-030-79840-6_4
Project(s): EMpowering People in deAling with internet of THings ecosYstems
Metrics:
Gallo S., Manca M., Mattioli A., Paternò F., Santoro C.
The rapid pervasive diffusion of Internet of Things technologies has opened up many opportunities for people to directly personalise the behaviour of surrounding objects and devices based on the dynamic events that can occur. To this end, several tailoring environments have been proposed supporting ... the enduser creation of trigger-action rules. Such tools can support different composition paradigms. In this paper we present a study that analyses three composition paradigms (graphical wizard, block-based, and conversational) to better understand how well they support rule creation activities. In order to make the analysis consistent we considered three implementations of such composition paradigms supporting the same set of triggers and actions. We have carried out a first user study in order to gather empirical feedback for substantiating our analysis, which provides indications of the pros and cons of each approach. [show more]DOI: 10.1007/978-3-030-79840-6_4
Project(s): EMpowering People in deAling with internet of THings ecosYstems
Metrics:
See at:
CNR IRIS | link.springer.com | ISTI Repository | ISTI Repository | CNR IRIS | CNR IRIS | CNR IRIS
2022
Conference article
Open Access
Towards understanding the transparency of automations in daily environments
Paternò F., Gallo S., Manca M., Mattioli A., Santoro C.
This paper outlines a proposal for how to address transparency of automations in daily environments, such as smart homes, based on experiences carried out in previous projects. The trigger-action programming paradigm has been used to describe and implement such automations in both commercial and res... earch tools. Such automations can be generated through machine learning techniques or directly by the end users or through an interaction between an intelligent agent and the user. When they are executed the resulting behaviour does not always result in the desired actions, and users may have difficulties in understanding and controlling them. Thus, there is a need for design criteria and associated tools that help people to understand and control what happens with the automations active in the environments where they live, and explain how they work and can be modified to better meet their needs. [show more]Source: CEUR WORKSHOP PROCEEDINGS, pp. 187-191. Helsinki, Finland, 21-22/03/2022
Paternò F., Gallo S., Manca M., Mattioli A., Santoro C.
This paper outlines a proposal for how to address transparency of automations in daily environments, such as smart homes, based on experiences carried out in previous projects. The trigger-action programming paradigm has been used to describe and implement such automations in both commercial and res... earch tools. Such automations can be generated through machine learning techniques or directly by the end users or through an interaction between an intelligent agent and the user. When they are executed the resulting behaviour does not always result in the desired actions, and users may have difficulties in understanding and controlling them. Thus, there is a need for design criteria and associated tools that help people to understand and control what happens with the automations active in the environments where they live, and explain how they work and can be modified to better meet their needs. [show more]Source: CEUR WORKSHOP PROCEEDINGS, pp. 187-191. Helsinki, Finland, 21-22/03/2022
See at:
ceur-ws.org | CNR IRIS | ISTI Repository | CNR IRIS