[SoC] GSoC 2023 - Final report - Implement SensorThings API standard in istSOS
Filippo Finke
filippo.finke at gmail.com
Fri Aug 25 04:00:00 PDT 2023
# Title and Organization
Project: Support basic SensorThings API (STA) for istSOS
Organization: Osgeo (Open Source Geospatial Foundation)
## Abstract
During the Google Summer of Code 2023, I worked on the project
"istsos-miu"[^1] part of the OSGeo[^2] Foundation. The goal of the project
was to create an implementation of the Sensor Things API[^3] standard for
the istsos project. The Sensor Things API is a standard created by the Open
Geospatial Consortium (OGC) for the Internet of Things (IoT). The standard
defines an open and unified way to interconnect IoT devices, data, and
applications over the Web. The standard is based on the RESTful
architecture style and uses the HTTP protocol for data transfer.
## The state of the art before GSoC
Before GSoC there was an implementation of the Sensor Things API standard
written in Java, but we wanted to create a new implementation in Python to
be able to integrate it with the istsos project. So the project was built
from scratch.
## The addition (added value) that your project brought to the software
Throughout the Google Summer of Code 2023, my primary objective was to
enhance the functionality of the istsos project by aligning it with the
standards set forth by the Sensor Things API[^3]. I successfully achieved
this goal by contributing to various components of the project, each of
which played a vital role in its development and improved usability. My
contributions can be categorized as follows:
### STA Language Parser and Lexer
One of the cornerstones of my work was the design and development of a
dedicated parser and lexer for the Sensor Things API (STA) language, which
I named "sta-parser"[^4]. This parser and lexer duo served as a fundamental
interface, enabling users to interact with the istsos server through the
STA language – a query language specifically tailored to the Sensor Things
API. The parser and lexer, packaged as a standalone solution, hold
potential for use in other projects beyond istsos as well. While the parser
is continuously evolving, it currently boasts support for all the essential
features of the STA language. This achievement opens the door to further
improvements, making the STA language more accessible and intuitive for
users.
### API Service Creation
A substantial portion of my efforts was dedicated to the creation of a new
API service within the istsos project. This service is pivotal in enabling
users to seamlessly interact with the istsos server by utilizing the
capabilities of the Sensor Things API. The newly developed API service
facilitates a wide array of operations, including:
- STA Language Queries: Users can now execute queries using the STA
language, providing a standardized way to request data from the istsos
server.
- Data Insertion: The API service empowers users to insert new data into
the system, streamlining the process of populating the database with
essential IoT data.
- Data Updates and Deletions: According to the STA standard, users can
effortlessly update and delete data entries using the API service, ensuring
data accuracy and relevancy.
This comprehensive API service acts as a bridge between users and the
istsos project, making it easier than ever to harness the power of the
Sensor Things API for effective IoT data management.
## The current state of the project
The goal of achiving an implementation of the Sensor Things API for the
istsos project has been achieved. The project is now in a state where it
can be used by users to interact with the istsos server using the Sensor
Things API. The project is still in development and will continue to be
developed in the future.
## Potential Future Work
While significant progress has been made during the Google Summer of Code
2023, certain features from the official Sensor Things API standard were
not able to be fully implemented within the given timeframe. These missing
features include:
### MQTT Extension:
Implementing the MQTT extension for seamless communication between IoT
devices and applications.
### Data Array Extension:
Incorporating support for data arrays to enable efficient storage and
retrieval of multiple data points.
### MultiDatastream Extension:
Introducing the MultiDatastream extension to accommodate scenarios
involving multiple data streams.
### Batch Request Requirement:
Developing the capability to handle batch requests efficiently.
### Historical Location Auto-Creation Requirement:
Addressing the auto-creation of historical location information, ensuring
accurate historical data representation.
### Pagination Requirement:
Implementing data pagination to manage and present large datasets
effectively.
### Built-in Query Functions Requirement:
Adding built-in query functions to simplify and enhance data querying
capabilities.
### Data Count Requirement:
Incorporating data count functionality for accurate monitoring of data
volume.
### Status Code Requirement:
Implementing support for querying data based on status codes, enhancing
data analysis possibilities.
Although these features could not be fully realized within the project
timeline, they represent potential areas for future development and
expansion of the "istsos-miu" project. Their implementation would further
enrich the project's capabilities and align it more closely with the
comprehensive Sensor Things API standard.
It's worth noting that while these features remain outstanding, the current
contributions have laid a strong foundation for their potential integration
in the future.
## What code has been merged
All the code that I have written during the Google Summer of Code 2023 has
been merged into the `traveltime` branch of the istsos project. The code
can be found in the following repositories:
- https://github.com/istSOS/istsos-miu
- https://github.com/filippofinke/sta-parser
The list of pull requests that have been merged can be found here:
-
https://github.com/istSOS/istsos-miu/pulls?q=is%3Apr+author%3Afilippofinke+is%3Aclosed
## What I have learned
Throughout the course of the Google Summer of Code 2023, I have gained a
wealth of knowledge and experience that has significantly enriched my
skills and understanding of software development, programming languages,
and open-source collaboration. The journey of contributing to the
"istsos-miu" project and aligning it with the Sensor Things API standard
has taught me invaluable lessons in various areas:
### Language Design and Parsing
I have learned a lot about language design principles and the intricate
process of creating parsers and lexers. Designing the "sta-parser" for the
Sensor Things API language required a deep dive into the world of parsing
techniques, grammars, and syntax analysis. This experience not only
sharpened my coding abilities but also honed my problem-solving skills in
tackling complex challenges related to parsing and interpreting
domain-specific languages.
### RESTful Architecture and API Design
Working on the "istsos-miu" project exposed me to the nuances of RESTful
architecture and API design. I gained insights into designing interfaces
that follow best practices for exposing functionalities over the web.
Crafting the API service to seamlessly integrate with the istsos server
using the capabilities of the Sensor Things API taught me the importance of
standardized communication and data exchange protocols in the context of
IoT.
### Collaboration and Open Source Development
Being part of an open-source project under the OSGeo Foundation has been an
incredible experience in collaborative software development. Through code
reviews, discussions, and interactions with mentors and other contributors,
I learned how to effectively collaborate in a remote and diverse
environment. The feedback received during the review process has immensely
improved the quality of my code and enhanced my ability to communicate
technical ideas clearly.
### Python Proficiency
While Python was not my primary language before this project, I have grown
significantly more proficient in it due to the nature of the tasks I
undertook. Developing the parser, lexer, and other components of the
project in Python expanded my programming language repertoire and gave me a
broader toolkit for future software endeavors.
### Project Management and Time Management
The Google Summer of Code experience required effective project management
and time management skills. Balancing the goals of the project, meeting
milestones, and overcoming unexpected challenges within the given timeframe
taught me how to prioritize tasks and manage my efforts efficiently.
### Technical Research and Adaptation
Implementing the Sensor Things API standard demanded a thorough
understanding of the technical specifications and principles behind it.
This required extensive research, learning, and adaptation of concepts
related to IoT, RESTful APIs, and data querying. The experience of diving
into unfamiliar territory and rapidly grasping new concepts has made me a
more adaptable and resourceful developer.
## Conclusion
As the Google Summer of Code 2023 comes to a close, I am filled with
gratitude for the incredible opportunities provided by Google and the OSGeo
organization. This journey of contributing to the "istsos-miu" project,
aligned with the Sensor Things API, has been a transformative experience.
Undoubtedly, the project's scope exceeded the initial proposal, hinting at
the potential for a more expansive endeavor. Nevertheless, the project has
achieved a level of functionality that aligns with the primary objectives
of implementing the Sensor Things API within the istsos project. Users can
now seamlessly interact with the istsos server using the standardized
interface, paving the way for efficient IoT data management.
I extend a heartfelt thank you to Google and the OSGeo organization for
providing me with this invaluable opportunity to contribute to open-source
development. My deepest appreciation goes to Massimiliano and Daniele,
whose mentorship, feedback, and support have been instrumental throughout
this journey. Their dedication and expertise have guided me in overcoming
challenges and pushing the project to new heights.
As I reflect on this experience, I recognize the substantial strides made
in enhancing my skills and understanding of language design, API
development, collaboration, and more. This journey has equipped me with the
tools to tackle complex technical challenges and collaborate effectively
within the open-source community.
While certain features of the Sensor Things API standard remain on the
horizon for future development, I am proud of the foundation we have built.
The merged code in the istsos project repositories stands as a testament to
the collective effort put into this endeavor.
As I look ahead, I am excited to continue contributing to the open-source
community and leveraging the lessons learned from this project to pursue
new opportunities.
Thank you once again to Google, the OSGeo organization, Massimiliano,
Daniele, and all those who have supported me on this remarkable journey.
## Image
## Links
- [Project Proposal](
https://summerofcode.withgoogle.com/proposals/details/QII7BFfJ)
- [Project Repository](https://github.com/istSOS/istsos-miu)
- [Project Wiki](
https://github.com/istSOS/istsos-miu/wiki/GSoC-2023-Implement-SensorThings-API-standard-in-istSOS-Filippo-Finke
)
- [OSGeo GSoC 2023](
https://wiki.osgeo.org/wiki/Google_Summer_of_Code_2023_Accepted)
[^1]: https://github.com/istSOS/istsos-miu
[^2]: https://www.osgeo.org/
[^3]: https://www.ogc.org/standard/sensorthings/
[^4]: https://github.com/filippofinke/sta-parser
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/soc/attachments/20230825/e1a7013d/attachment.htm>
More information about the SoC
mailing list