Skip to content

Commit be0cf74

Browse files
mhthiesmhthies
committed
Add documentation for MySQL and InMemoryDataLogVariable
1 parent 4e49250 commit be0cf74

File tree

4 files changed

+163
-1
lines changed

4 files changed

+163
-1
lines changed

‎docs/data_logging.rst

+10-1
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,8 @@ Typically, `DataLogVariables` are used to add :ref:`Log Widgets <web.log_widgets
1919
Included interfaces with data logging support
2020
---------------------------------------------
2121

22-
TODO
22+
* :class:`shc.interfaces.mysql.MySQLConnector`
23+
* :class:`shc.interfaces.in_memory_data_logging.InMemoryDataLogVariable`
2324

2425

2526

@@ -41,3 +42,11 @@ TODO
4142
.. autoclass:: LoggingWebUIView
4243
:show-inheritance:
4344
:members:
45+
46+
``shc.interfaces.in_memory_data_logging`` Module Reference
47+
----------------------------------------------------------
48+
49+
.. automodule:: shc.interfaces.in_memory_data_logging
50+
51+
.. autoclass:: InMemoryDataLogVariable
52+
:show-inheritance:

‎docs/interfaces/mysql.rst

+70
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
2+
MySQL Database Interface
3+
========================
4+
5+
The MySQL interface allows to using a MySQL (or MariaDB or Percona) server for :ref:`data_logging` and/or persistence.
6+
7+
Setup
8+
-----
9+
10+
The MySQL interface does (currently) not include automated database setup or database schema migration, so the database needs to be set up manually.
11+
Typically, this includes chosing a secure password for the database connection and running the following commands in a mysql console with root privileges:
12+
13+
.. code-block:: mysql
14+
15+
CREATE DATABASE 'shc';
16+
CREATE USER 'shc'@'localhost' IDENTIFIED BY '<password>';
17+
GRANT SELECT, INSERT, UPDATE ON shc.* TO 'shc'@'localhost';
18+
FLUSH PRIVILEGES;
19+
20+
USE shc;
21+
22+
CREATE TABLE log (
23+
name VARCHAR(256) NOT NULL,
24+
ts DATETIME(6) NOT NULL,
25+
value_int INTEGER,
26+
value_float FLOAT,
27+
value_str LONGTEXT,
28+
KEY name_ts(name, ts)
29+
);
30+
31+
CREATE TABLE `persistence` (
32+
name VARCHAR(256) NOT NULL,
33+
ts DATETIME(6) NOT NULL,
34+
value LONGTEXT,
35+
UNIQUE KEY name(name)
36+
);
37+
38+
Usage
39+
-----
40+
41+
With the database setup listed above, the database can be used in an SHC project with the following code (or similar)::
42+
43+
import shc
44+
import shc.interfaces.mysql
45+
46+
# you may want to load the database password from some configfile to keep it out of your project code
47+
mysql_interface = shc.interfaces.mysql.MySQLConnector(host="localhost", db="shc", user="shc", password="<password>")
48+
49+
# ...
50+
51+
# a variable with logging
52+
my_room_temperature = shc.Variable(float, "my_room_temperature") \
53+
# example for connecting to the real world:
54+
#.connect(mqtt_interface.topic_string("temp/my_room"), convert=true) \
55+
.connect(mysql_interface.variable(float, "my_room_temperature"))
56+
57+
# a variable with persistence
58+
my_room_temperature = shc.Variable(bool, "temperature_control_active") \
59+
.connect(mysql_interface.persistence_variable(bool, "temperature_control_active"), read=true)
60+
61+
62+
Module Documentation
63+
--------------------
64+
65+
.. automodule:: shc.interfaces.mysql
66+
67+
.. autoclass:: MySQLConnector
68+
69+
.. automethod:: variable
70+
.. automethod:: persistence_variable

‎shc/interfaces/in_memory_data_logging.py

+14
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,20 @@
77

88

99
class InMemoryDataLogVariable(Writable[T], DataLogVariable[T], Readable[T], Generic[T]):
10+
"""
11+
A single in-memory :class:`DataLogVariable <shc.data_logging.DataLogVariable>`, based on a simple Python list,
12+
without any persistent storage.
13+
14+
Data log entries that are older than the specified `keep` time are dropped automatically, to keep memory usage
15+
limited. This class is sufficient for logging a variable value for the purpose of displaying a chart over the last
16+
few minutes (or maybe hours) and calculate aggregated values, if you don't mind losing the historic data on every
17+
restart of the SHC application. It's also fine for demonstration purposes (see ui_logging_showcase.py in the
18+
examples/ directory).
19+
20+
:param type\_: Value type of this `connectable` object (and as a `DataLogVariable`)
21+
:param keep: timespan for which to keep the values. Older values will be deleted upon the next `write` to the
22+
object.
23+
"""
1024
type: Type[T]
1125

1226
def __init__(self, type_: Type[T], keep: datetime.timedelta):

‎shc/interfaces/mysql.py

+69
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,50 @@
1818

1919

2020
class MySQLConnector(ReadableStatusInterface):
21+
"""
22+
Interface for using a MySQL (or MariaDB or Percona) server for logging and/or persistence
23+
24+
A database with the following schema needs to be created manually on the database server:
25+
26+
.. code-block:: mysql
27+
28+
CREATE TABLE log (
29+
name VARCHAR(256) NOT NULL,
30+
ts DATETIME(6) NOT NULL,
31+
value_int INTEGER,
32+
value_float FLOAT,
33+
value_str LONGTEXT,
34+
KEY name_ts(name, ts)
35+
);
36+
37+
CREATE TABLE `persistence` (
38+
name VARCHAR(256) NOT NULL,
39+
ts DATETIME(6) NOT NULL,
40+
value LONGTEXT,
41+
UNIQUE KEY name(name)
42+
);
43+
44+
For data logging of all value types that are derived from `int` (incl. `bool`), `float` or `str`, the respective
45+
`value\_` column is used. This includes Enum types with a value base type in any of those. Otherwise, the value is
46+
JSON-encoded, using SHC's generic JSON encoding/decoding system from the :mod:`shc.conversion`, and stored in the
47+
`value_str` column.
48+
49+
Values of persistence variables are always JSON-encoded for storage.
50+
51+
All timestamps are stored in UTC.
52+
53+
The given constructor keyword arguments are passed to
54+
`aiomysql.connect() <https://aiomysql.readthedocs.io/en/latest/connection.html#connection>`_ for configuring the
55+
connection to the database server:
56+
57+
* ``host: str`` (default: ``"localhost"``)
58+
* ``port: int`` (default: ``3306``)
59+
* (optional) ``unix_socket: str``
60+
* (optional) ``user: str``
61+
* ``password: str`` (default: ``""``)
62+
* ``db: str`` – name ot the database
63+
* (optional) ``read_default_file`` – path to my.cnf file to read all these options from
64+
"""
2165
def __init__(self, **kwargs):
2266
super().__init__()
2367
# see https://aiomysql.readthedocs.io/en/latest/connection.html#connection for valid parameters
@@ -51,6 +95,19 @@ async def _get_status(self) -> "InterfaceStatus":
5195
return InterfaceStatus(ServiceStatus.OK, "")
5296

5397
def variable(self, type_: Type[T], name: str) -> "MySQLLogVariable[T]":
98+
"""
99+
Creates a `connectable` object with the given value type for logging a time series of the given name in the
100+
MySQL database
101+
102+
The returned object is :class:`writable <shc.base.Writable>`, :class:`readable <shc.base.Readable>` and a
103+
(subscribable) :class:`DataLogVariable <shc.data_logging.DataLogVariable>`.
104+
105+
:param type_: The value type of the returned connectable variable object. It must provide a
106+
:ref:`default JSON serialization <datatypes.json>`.
107+
:param name: The name of the time series in the database. Its length must not exceed the declared maximum length
108+
of the log.name column in the database schema (256 by default). If the same name is requested
109+
twice, the *same* connectable object instance will be returned.
110+
"""
54111
if name in self.variables:
55112
variable = self.variables[name]
56113
if variable.type is not type_:
@@ -63,6 +120,18 @@ def variable(self, type_: Type[T], name: str) -> "MySQLLogVariable[T]":
63120
return variable
64121

65122
def persistence_variable(self, type_: Type[T], name: str) -> "MySQLPersistenceVariable[T]":
123+
"""
124+
Creates a `connectable` object with the given value type for persisting (only) the current value of a connected
125+
object under the given name in the MySQL database
126+
127+
The returned object is :class:`writable <shc.base.Writable>`, :class:`readable <shc.base.Readable>`.
128+
129+
:param type_: The value type of the returned connectable object. It must provide a
130+
:ref:`default JSON serialization <datatypes.json>`.
131+
:param name: The *name* for identifying the value in the database. Its length must not exceed the declared
132+
maximum length of the log.persistence column in the database schema (256 by default). If the same
133+
name is requested twice, the *same* connectable object instance will be returned.
134+
"""
66135
if name in self.persistence_variables:
67136
variable = self.persistence_variables[name]
68137
if variable.type is not type_:

0 commit comments

Comments
 (0)