I need some design help with a database and how to process the data. I’m working on my first major project and extremely excited about it. All my knowledge about programming and database is self taught and I’m ruing into some roadblocks because I’m not sure what’s the best way to accomplish this.
What I’m doing is I have a power meter (Neurio) for my house and I want to log the data into a MySQL database. The data refreshes every second and outputs in JSON and I’m able to write the data to the database using Python. I want to save the data every second for the first hour and after the first hour consolidate the data down to 5 seconds (to cut the size down for a reduce the data that is needed to be requested.) After the day I want to condense it even more, down to 1 minute. Then after that, at the 1 month mark consolidate it down to an hour. I have no idea on how to accomplish this... if MySQL is able to consolidate this or I have to have python do this then rewrite the data to the database.
Here is a sample of the JSON output for one second.
{
"sensorId": "XXXXXXXXXXXXXXXX",
"timestamp": "2016-04-26T23:10:30Z",
"chaels": [
{
"type": "PHASE_A_CONSUMPTION",
"ch": 1,
"eImp_Ws": 1794727517,
"eExp_Ws": 212128,
"p_W": 2226,
"q_VAR": 258,
"v_V": 120.545
},
{
"type": "PHASE_B_CONSUMPTION",
"ch": 2,
"eImp_Ws": 2592781259,
"eExp_Ws": 18,
"p_W": 2481,
"q_VAR": 706,
"v_V": 120.536
},
{
"type": "CONSUMPTION",
"ch": 3,
"eImp_Ws": 4387296670,
"eExp_Ws": 34,
"p_W": 4707,
"q_VAR": 965,
"v_V": 120.541
}
],
"cts": [
{
"ct": 1,
"p_W": 2226,
"q_VAR": 258,
"v_V": 120.545
},
{
"ct": 2,
"p_W": 2481,
"q_VAR": 706,
"v_V": 120.536
},
{
"ct": 3,
"p_W": 0,
"q_VAR": 0,
"v_V": 0.130
},
{
"ct": 4,
"p_W": 0,
"q_VAR": 0,
"v_V": 120.544
}
]
}
I’m not sure if I should write all this data to one table or have multiple tables… I was thinking about using multiple tables to accomplish this, But at this time I’m not sure how to use foreign keys. What do you think the best way is to layout this table and be able to condense the data down at those time intervals?
I plan on making edits to this question for the next hour or so as I come across more issues along the line or thought on how to go about what I want to do, but I figured I would get the question out there and maybe your answers to my question would help me correct and mistakes or thoughts.
Thank you all for taking the time to read my question and hopefully answer it in anyway possible, Throdne
