I am currently evaluating possible NoSQL technologies for my usecase. I have a lot of sensor data logs (several TBs) and want to analyze the data in a scalable, efficient way.
My sensor data contains a timestamp, Location, and a List of Sensor attribute name (for example "temperature") and Sensor attribute value (for example 12.9°C). However, the location is not present in all of my data entries. Also, the list of attributes per entry is varying. So one log entry could contain data like this:
- sensor_id: 1
- location: lat/lon (note: but not always included in each entry)
- datetime: timestamp_of_log_date
- attribute_name1: "attribute_value1"
- attribute_name2: 123123
- attribute_name3: 0.933
- attribute_nameN: "attribute_valueN"
I recently came across graph databases and find the concept interesting. I am not sure whether my usecase should be applied on a graph database because of (maybe) lacking relationships between my data.
There are some ways to model a relationship between my sensor data but right now I can only think of the following relationships:
- Date/Time of the entries to Year->Month->Day->Hour->Minute->Second
- Location of the entries to Continent->Country->State->City->Street/PoI
- Sensor ID->Attribute names
Does that even make sense? ;-)
One more thing to note: When I'm ruing geospatial queries, I usually want to read out further records based on the timestamp of the record that was found in the spatial query. So basically, when my geospatial query finished and retus a record with timestamp "25.06.2016-20:40:30" I also want to read out ALL other records that have a timestamp within one minute for each record found in the geospatial query.
Neo4j for example is good at processing geospatial queries, which is a pretty common query type for my usecase. But how would I model my data? How are my nodes coected to the other nodes? Do they even need to be coected?