Prof. Siniša Srbljić
Our world is faced with a number of environmental, social, political, economical, energy, and scientific challenges. We believe that our ability to face these challenges will be greatly affected by how well and soon we can master the scalability, sustainability, and equality of the digital world – a world of collaborating humans and computers that combines the communication and computing capabilities of machines with the creativity and high-level cognitive capabilities of people. Digital world sustainability and scalability raise many technical, political, ethical, and economical issues as the scale, scope, and connectivity of these human-computer networks increases. Two main processes emerge from the worldwide effort to create sustainability, scalability and equality in the digital world. The first process fosters people around the world to become digital world participants and contributors constituting collective intelligence and consciousness – just two out of six billon people have this privilege today. The second process promotes digital world contributors to become digital world developers – at most forty million app developers enjoy this privilege. While the first process relies on social, economical, political, and educational aspects, the second process mostly depends on technological progress.
As a premise of our research, we believe that digital world scalability, sustainability, and equality cannot be achieved without technology that will turn each and every digital world participant into a digital world designer and developer. We organize the Digital World Design and Development ecosystem into three main parts: App Composers, Consumer Assistants, and App Designer Communities of Interest.
App Composers
App Composers provide a digital world consumer-proof app design and development environment and methodology. In the early phase of our research (2004-2008) while mentoring Daniel Skrobo (Service Composition Based on Tabular Programming, Ph.D. dissertation, FER-UNIZG, 2008), we developed the HUSKY language for service composition – a semi-graphical language based on a combination of coordination and scripting languages. HUSKY combines the features of textual and visual languages in a tabular representation. We reused basic spreadsheet concepts, such as cells, cell values, and cell references, and extended them with declarations of domain specific and system services, as well as imperative constructs for their invocations (Srbljic, Skvorc, Popovic: Programming Languages for End-User Personalization of Cyber-Physical Systems“, AUTOMATIKA, 2012).
However, we found that service-oriented methodology is not appropriate for app design and development, because services and APIs are not semantically intuitive and easy to use for non-programmers and non-mathematicians. Our idea was to replace APIs with GUIs that are more intuitive and friendly to use for digital world consumers. After discussions with Boris Debic from Google (2006) regarding Google Gadgets, we shifted our focus from service-oriented to widget-oriented technology (From Service to Gadget Composition: Enabling Process-oriented Web, Google Tech Talk, 2007). Software gadgets or widgets with their GUIs are semantically intuitive and friendly for use by digital word consumers. While conducting research on the appropriate app design and development environment, we assigned widgets with a threefold role: they are the basic building blocks for app development, the apps developed from widgets are again in the form of widgets, and widgets themselves are again used to implement tools for app design and development (Geppeto: Promoting End-users to Gadget Developers, Google Tech Talk, 2008). While mentoring Dejan Skvorc (Consumer Programming, Ph.D. dissertation, FER-UNIZG, 2010), we based our digital world consumer programming model on a widget-oriented architecture, GUI level language, programming by demonstration, and semi-graphical tabular representation. To prove this concept, we developed Geppeto (http://www.geppeto.fer.hr/) – a consumer programming platform. Geppeto provides consumer-programmable widgets which enable consumers to integrate a set of application-specific widgets into a new composed widget (Geppeto: Consumer’s Approach to Programming, Google Tech Talk, 2008).
In order to run composed widgets, we developed a web browser-based execution engine for multitask execution of consumer apps. The execution engine provides operating system kernel mechanisms within a web browser, such as scheduling, asynchronous communication through message queues, and semaphore-based synchronization. Part of this research was conducted in cooperation with Google’s Gadgets group in Mountain View, where Dejan Skvorc and Ivan Zuzak developed a framework for inter-widget communication (Skvorc, Zuzak: The Publish-Subscribe Framework for Inter-Widget Communication, Google Engineering Internship Program, 2007).
Since execution of widget-based Web systems should be in line with the software architectural principles underpinning the World Wide Web, Ivan Zuzak continued this research as a part of his Ph.D. dissertation under the working title A Finite State Machine Meta-model for Describing Hypermedia Systems Based on the REST Architectural Style. Ivan Zuzak is studying and modeling these principles in order to advance the design of software frameworks for implementing Web systems and improving the compatibility of implemented systems with those principles.
Besides the app composition widget, Geppeto implements other programmable widgets that enable building of time-driven and event-driven apps (Srbljic, Skvorc, Skrobo: Widget-Oriented Consumer Programming, AUTOMATIKA, 2010), location-driven apps, communicating apps, and distributed apps (http://www.geppeto.fer.hr/). While mentoring Miroslav Popovic (Consumer Program Synchronization, Ph.D. dissertation, FER-UNIZG, 2011), we designed a set of consumer-oriented widgets for synchronization. These synchronization widgets are suitable for use in consumer social networks, consumer programs, and engineered software systems. Mechanisms of the designed widgets resemble social processes in order to be intuitive and easy to use for consumers.
In addition to generic programmable widgets, we research application-specific widgets in different application domains. In co-mentoring Tomislav Lugaric on his Ph.D. thesis under the working title Application of Consumer Programming in Underwater Robotics with professor Zoran Vukic from the Laboratory for Underwater Systems and Technologies, University of Zagreb, Faculty of Electrical Engineering and Computing, we are researching the application of consumer programming in areas such as smart ROV control and autonomous underwater demining. In our research, vehicles, sensors, control blocks, and additional processing blocks are represented to the user as widgets that can be composed to form a complex app used to guide the vehicle through the mission. Consumer programming will enable faster on-site reconfiguration of mission control software and easier reaction to change or modification of a vehicle, thus improving the quality of experience for users of unmanned underwater and surface vehicles.
While mentoring Zvonimir Pavlic (Consumer-oriented Widgets for Statistical Processing, MEng thesis, FER-UNIZG, 2010), we developed a distributed system of consumer-oriented widgets for statistical analysis. The system implements functionalities of data retrieval, statistical processing, displaying the results, and defining events and actions. While mentoring Tomislav Lugaric (Consumer-oriented Widgets for Finance Management System, MEng thesis, FER-UNIZG, 2010), we developed a distributed system of gadgets suited for financial analysis. The system was developed in cooperation with students of Faculty of Economics and Business and allowed the consumer to interact with data from Zagreb Stock Exchange and Google Finance.
In cooperation and by invitation from NOAA (National Oceanographic and Atmospheric Administration, Pacific Grove, CA), we developed a set of application specific widgets for data management and analysis of climate and oceanic data sets collected, processed and published by NOAA.
Consumer Assistants
Consumer Assistants implement wizards that use crowdsourcing and collective intelligence methods to help consumers during app design and development.
As consumer apps are composed out of diverse widgets, every app has its inherent internal structure. In mentoring Ivan Budiselic on his Ph.D. thesis under the working title Structure-Based Recommender System for Consumer Programming, we are analyzing this structure in a larger collection of apps and extracting usage patterns that expose implicit relationships between widgets. These relationships then form a basis for a recommender system that helps consumers build their apps by suggesting potentially useful widgets based on the context of a partially completed composition.
Building components for consumer apps are often not under control of the actual app developer, but are usually used as provided by other developers. Building apps in such a volatile environment raises inevitable reliability issues. In mentoring Goran Delac on his Ph.D. thesis under the working title Fault Tolerant Consumer Programming, the goal of our research is to develop a framework that can be used by the consumer to construct an app with a desired level of reliability. To reason about the overall reliability of consumer apps, we will construct a probabilistic graphical model. The model will be used to recommend the points of improvement in the app with respect to achieved reliability growth and spent system resources.
The perceived quality of a complex consumer application depends on the functional and non-functional attributes of the composed widgets. Hence, a consumer selecting the widgets to be composed should ideally have reliable information regarding the candidate widgets’ provided functionalities and their nonfunctional qualities (e.g., security, availability, reliability). In mentoring Marin Silic on his Ph.D. thesis under the working title A Formal Model for Prediction of Consumer Computational Elements Availability, we are focusing our research on widgets’ nonfunctional qualities. In order to estimate the actual nonfunctional qualities of widgets’ in consumer programming, we use prediction models that include as many factors as possible which may impact consumer widgets’ performance.
Complex consumer apps are composed out of different kind of widgets representing objects from both cyber and physical worlds. The behavior of each widget follows the laws of the world it comes from. Interaction of widgets from different worlds imposes sophisticated composition rules and procedures. In mentoring Zvonimir Pavlic on his Ph.D. thesis under the working title Cyber-Physical Coexistence in Consumer Computing, we are researching recommenders that both help consumers to compose valid widgets that follow laws from different worlds and provide sound implementation of a composed widget. It is essential to successfully close the control loop between the cyber and physical objects, so that consumer actions in the virtual world have the concrete real-time consequences in the physical world via the widget-managed physical device. Therefore, we are researching possibilities of consumer programs’ optimization and translation to the lower levels of the control stack.
App Designer Communities of Interest
App Designer Communities of Interest provide a place where consumers, who share a common interest in app design and development, exchange information, ask questions, and express their opinions about app design and development problems. Consumers take part in the App Designer Communities of Interest to exchange information, to obtain answers to design problems, to improve their understanding, to share common knowledge and expertise, or to collectively participate in the app design and development process. Through the process of sharing information and experiences within the group, consumers learn from each other, and have an opportunity to develop themselves personally in the area of app design and development. In App Designer Communities of Interest, consumers are gathered together either by exchanging information through personal contacts or through social networks, or by assembling through special wizards that manage consumers expertise and knowledge in the areas of app design and development.
In mentoring Klemo Vladimir on his Ph.D. thesis under the working title Knowledge Management in Consumer-oriented Programming Communities, we are currently researching a methodology for implementation of wizards, which would help gather consumers in groups of common interest. The focus of our research are models, algorithms, and evaluation of mechanisms for locating the expertise and knowledge in the App Designer Communities of Interest with the goal of finding the best possible match between consumers that need specific knowledge and consumers that have the expertise required to solve the given app design and development problem.
Theoretical background
Since consumer computing relies on large sets of data about consumers and processes that represent consumer apps, we focus our research in theoretical problems such as big data incompleteness resolution by using statistical prediction models (Marin Silic), feature augmentation and evaluation of big data complex structure by using probabilistic graphical models (Goran Delac), pattern analysis of big data complex structure by using reasoning models (Ivan Budiselic), discovering of semantic relatedness within big data by using correlation models of social and process structures (Klemo Vladimir), and big data constraint analysis by using cyber-physical contextualization models (Zvonimir Pavic).
