diff --git a/docs/node/clients/python_wrapper.md b/docs/node/clients/python_wrapper.md new file mode 100644 index 0000000..ae2faf5 --- /dev/null +++ b/docs/node/clients/python_wrapper.md @@ -0,0 +1,525 @@ +# Python-Wrapper Documentation: + +:::note + +All of the documentation and testing done regarding the VILLASnode Python-wrapper +is based upon the `signal_v2` node-type provided by VILLASnode. +Node specific functions are implemented on a node to node basis and +therefore may exert different behavior. + +- [VILLASnode functions exposed by the C-API](#villasnode-functions-exposed-by-the-c-api) + - [Functions to set up a node or modify its state](#functions-to-set-up-a-node-or-modify-its-state) + - [Functions to extract node specific information](#functions-to-extract-node-specific-information) + - [Functions related to data transfer](#functions-related-to-data-transfer) +- [Installation](#installation) + + +## VILLASnode functions exposed by the C-API + +The C-API functions can be found [here](https://github.com/VILLASframework/node/blob/master/include/villas/node.h) + + +### Functions to set up a node or modify its state + + +`node_new(const char *id_str, const char *json_str)` takes two strings as input parameters +- `id_str:` identification string (uuid - 36 characters long + 1 character for null termination) + If not provided, resulting in the nullptr, a random uuid is created and assigned to the node by VILLASnode. + It can be retrieved by different functions like `node_name_full()`. + +- `json_str:` the string containing a valid json configuration + +Invalid json configurations will throw an error. +The required json format to configure nodes can be found [here](https://villas.fein-aachen.org/docs/node/nodes/). + +***Paths or the VILLASdaemon are not used for the Python-Wrapper instance. +Therefore only node configurations need to be considered.*** + +
+ Creating a Node + +```python +import uuid +import villas_node as vn + +# some valid json config +config = { + ... +} + +# config is a singular node configuration +# creating an uuid - optional +id = str(uuid.uuid4()) + +#invalid uuid's are discarded and a new random one is generated by VILLASnode +#node = vn.node_new(config) does not work, some input for the uuid is necessary +#node = vn.node_new("", config) +#node = vn.node_new("0", config) + +node = vn.node_new(id, config) + +# config is a list of node configurations +nodes = {} +# creating new nodes, accessible by name +for name, content in data.items(): + #dictionary to extract the name of each node + obj = {name: content} + + # read inner configuration/json object to create a node + config = json.dumps(obj, indent=2) + id = str(uuid.uuid4()) + + nodes[name] = vn.node_new(id, config) + +``` +
+ + +`int node_check(vnode *n)` checks the in and output signals of a node and sets the node state to **checked** + +`int node_prepare(vnode *n)` sets up the in and output signals of a node and sets the node state **prepared** + +`int node_start(vnode *n)` starts a node and sets the node state **started** + +`int node_stop(vnode *n)` stops a node, can be (re-)started and but resumed + +`int node_pause(vnode *n)` pauses a node, can be resumed and stopped + +`int node_resume(vnode *n)` resumes a paused node, does not work if the node stopped + +`int node_restart(vnode *n)` restarts a stopped node + + +`int node_destroy(vnode *n)` deletes/destroys a node, can not be used again leaving the node pointer dangling (do not use (the pointer)) - in this case the variable assigned to the parameter `vnode *n` + +- `vnode *n` a node pointer that can be created by [node_new()](#node_new()) + +The functions have return codes on success or failure. +In other words either `-1`, `0`, `1` is returned depending on either: +- unchanged `-1` +- success `0` +- failure `1` + +An example: in case `node_start()` is used on a node that has already been started `1` is returned. +This can be used for branching. +
+ All of these functions can be used like this + +```python +# assuming node is a valid node +node = ... + +node_prepare(node) +node_check(node) +status = node_start(node) +node_stop(node) +node_pause(node) +node_resume(node) +node_restart(node) +node_destroy(node) + +# branching +if (status == -1): + print("Node already started!") +if (status == 0): + print("Node started!") +if (status == 1): + print("Starting the node failed!") +``` +
+ + +### Functions to extract node specific information + +`bool node_is_valid_name(const char *name)` + +`bool node_is_enabled(const vnode *n)` + +`const char *node_name(vnode *n)` returns the node name + +`const char *node_name_short(vnode *n)` currently not working + +`const char *node_name_full(vnode *n)` returns name and details of the node +>output structure of the details: +node_name\: uuid=\, +#in.signals=<.../...>, +#out.signals<.../...>, +#in.hooks=..., +#out.hooks=..., +in.vectorize=..., +out.vectorize=..., +out.netem=..., +layer=..., +in.address=, +out.address + +`const char *node_details(vnode *n)` returns less details than node_name_full() +>layer=..., +in.address=\, +out.address=\ + +`unsigned node_input_signals_max_cnt(vnode *n)` returns input signal count + +`unsigned node_output_signals_max_cnt(vnode *n)` returns output signal count + +`const char *node_to_json_str(vnode *n)` returns node config in string format + +`unsigned sample_length(vsample *smp)` returns the length of the samples stored in a sample object + +
+ All of these functions can be used like this + +```python +# assuming node is a valid node +name = "some name" +node = ... + +node_is_valid_name(name) +node_is_enabled(node) +node_name(node) +node_name_short(node) +nodfe_name_full(node) +node_details(node) +node_input_signals_max_cnt(node) +node_output_signals_max_cnt(node) +node_to_json_str(node) + +# sample_length() requires a sample handle +#this can be a sample stored in an array +samples = smps_array(1) +samples[0] = sample_alloc(i) #i should be the sample length +... + +sample_length(samples[0]) + +# or a sample created manually with sample_pack() + +sample = sample_pack(...) + +sample_length(sample) +``` +
+ +`json_t *node_to_json(const vnode *n):` returns a node configuration of the node + +The node configuration returned is either of type: +- `none` +- `int` +- `float` +- `bool` +- `string` + +the following returned types contain the ones above: +- `dictionary` +- `list` + + +### Functions related to data transfer + +`smps_array(int size)` fixed size data structure to hold samples + +Before samples can be stored within the sample array, each sample has to be allocated. +The index starts with 0 instead of 1 and ends with `len(smps_array) - 1` as would be usual in a system programming languages. +Only set and get functions are provided and can be accessed by the `=` operator. +When assigning a new sample to an already existing sample within the data structure, it automatically handles deallocation. Ideally it is not necessary to ever call `sample_decref()`. +Once sample entries within the array are allocated they do not need to be reallocated. + +`int node_reverse(vnode *n)` swap in and output signals of a node + +`int node_read(vnode *n, vsample **smps, unsigned cnt)` reads from the node's storage/buffer + +- `vnode *n` the pointer to a node +- `vsample **smps` is a pointer to a data structure that can hold samples + + Since this is impossible to do, without a wrapper class such as the **sample holding array**, natively with a python data structure, the **samples array** has to be used for this. + +- `unsigned cnt` the amount of samples to read + + Some node-types like the `signal_v2` node can only **read** one sample at a time. + Considering rt-mode for the `signal_v2` node, trying to read the next sample before it is ready will result in the program stalling till the next sample is generated. + The same applies for **reading** from a socket before the socket has received any samples. In this case the thread trying to read from the socket will lock up. + +`int node_write(vnode *n, vsample **smps, unsigned cnt)` writes to a node's storage/buffer + +The same as `node_read()` above except for the waiting/locking up. + +`int node_poll_fds(vnode *n, int fds[])` + +`int node_netem_fds(vnode *n, int fds[])` + +`vsample *sample_alloc(unsigned len)` allocates a single sample + +`void sample_decref(vsample *smp)` decrement and delete sample pointer + +The sample is deleted and deallocated if it has no pointers pointing to it. + +`vsample *sample_pack(unsigned seq, struct timespec *ts_origin, + struct timespec *ts_received, unsigned len, + double *values)` creates a sample manually + +`void sample_unpack(vsample *s, unsigned *seq, struct timespec *ts_origin, + struct timespec *ts_received, int *flags, unsigned *len, + double *values)` + +`int memory_init(int hugepages)` initializes memory system with **hugepages** + + +
+ configuration file for the following example + +```json +{ + "send_socket": { + "type": "socket", + "format": "protobuf", + "layer": "udp", + "in": { + "address": "127.0.0.1:65532", + "signals": [ + { + "name": "voltage", + "type": "float", + "unit": "V" + }, + { + "name": "current", + "type": "float", + "unit": "A" + } + ] + }, + "out": { + "address": "127.0.0.1:65533", + "netem": { + "enabled": false + }, + "multicast": { + "enabled": false + } + + } + }, + "intmdt_socket": { + "type": "socket", + "format": "protobuf", + "layer": "udp", + "in": { + "address": "127.0.0.1:65533", + "signals": [ + { + "name": "voltage", + "type": "float", + "unit": "V" + }, + { + "name": "current", + "type": "float", + "unit": "A" + } + ] + }, + "out": { + "address": "127.0.0.1:65534", + "netem": { + "enabled": false + }, + "multicast": { + "enabled": false + } + } + }, + "recv_socket": { + "type": "socket", + "format": "protobuf", + "layer": "udp", + "in": { + "address": "127.0.0.1:65534", + "signals": [ + { + "name": "voltage", + "type": "float", + "unit": "V" + }, + { + "name": "current", + "type": "float", + "unit": "A" + } + ] + }, + "out": { + "address": "127.0.0.1:65535", + "netem": { + "enabled": false + }, + "multicast": { + "enabled": false + } + } + }, + "sig_gen_file" :{ + "type": "file", + "format": "villas.human", +"uri": "/path/to/sig_gen.log", + "in": { + "epoch_mode": "wait", + "signals": [ + { + "name": "voltage", + "type": "float", + "unit": "V" + }, + { + "name": "current", + "type": "float", + "unit": "A" + } + ] + } + }, + "recv_socket_file" :{ + "type": "file", + "format": "villas.human", + "uri": "/path/to/recv_socket.log", + "in": { + "epoch_mode": "wait", + "signals": [ + { + "name": "voltage", + "type": "float", + "unit": "V" + }, + { + "name": "current", + "type": "float", + "unit": "A" + } + ], + "hooks": [ + { + "type": "print", + "format": "villas.human" + } + ] + } + }, + "signal_generator": { + "type": "signal.v2", + "limit": 100, + "rate": 10, + "in": { + "signals": [ + { + "amplitude": 2, + "name": "voltage", + "phase": 90, + "signal": "sine", + "type": "float", + "unit": "V" + }, + { + "amplitude": 1, + "name": "current", + "phase": 0, + "signal": "sine", + "type": "float", + "unit": "A" + } + ], + "hooks": [ + { + "type": "print", + "format": "villas.human" + } + ] + } + } +} +``` +
+
+ Example code taken from the Wrapper Unit tests and slightly modified: + + +```python +import json +import uuid +import villas_node as vn + +# the configuration comprises nodes with the type and name: +# +# signal generator node (v2): "signal_generator" +# socket nodes: "send_socket", "intmdt_socket", "recv_socket" +# file nodes: "sig_gen_file", "recv_socket_file" + +with open('/path/to/config/file.json', 'r') as f: + data = json.load(f) + f.close() + +# list to read and create multiple nodes from a file +test_nodes = {} +for name, content in data.items(): + #dictionary to extract the name of each node + obj = {name: content} + + # forward inner configuration to create a node + config = json.dumps(obj, indent=2) + id = str(uuid.uuid4()) + + #creating new nodes, accessible by name + test_nodes[name] = vn.node_new(id, config) + +# verifying the node configurations and starting them +for node in test_nodes.values(): + if (vn.node_check(node)): + raise RuntimeError(f"Failed to verify node configuration") + if (vn.node_prepare(node)): + raise RuntimeError(f"Failed to verify {vn.node_name(node)} node configuration") + vn.node_start(node) + +# declare Arrays that can hold 1, 100 and 100 samples respectively +send_smpls = vn.smps_array(1) +intmdt_smpls = vn.smps_array(100) +recv_smpls = vn.smps_array(100) + +for i in range(0,100): + # allocate memory for samples to be stored with two signal values per Sample + send_smpls[0] = vn.sample_alloc(2) + intmdt_smpls[i] = vn.sample_alloc(2) + recv_smpls[i] = vn.sample_alloc(2) + + # generate signals and send over send socket, write to file + # signal nodes can only create one Sample at a time + vn.node_read(test_nodes["signal_generator"], send_smpls, 1) + vn.node_write(test_nodes["send_socket"], send_smpls, 1) + vn.node_write(test_nodes["sig_gen_file"], send_smpls, 1) + +# write intermediary signals to file (100 at once) +vn.node_read(test_nodes["intmdt_socket"], intmdt_smpls, 100) +vn.node_write(test_nodes["intmdt_socket"], intmdt_smpls, 100) + +# write receive socket signals to file (100 at once) +vn.node_read(test_nodes["recv_socket"], recv_smpls, 100) +vn.node_write(test_nodes["recv_socket_file"], recv_smpls, 100) +``` +
+ + +### Installation + +There are two recommended methods to install the VILLASnode Python-Wrapper. + + +1. Using one of the compatible Docker Containers which can be found [in the VILLASnode repository](https://github.com/VILLASframework/node/tree/python-wrapper/packaging/docker) + or can be installed [as is described here](../../installation.md). The Fedora container, which is also the development + container would be recommended first and foremost. +2. Build VILLASnode from source [as is described here](../installation.md) and make sure to have all of the necessary + dependencies installed. + +**The requirements for the Python-Wrapper differ from the versions listed in 2.** + +| Package | Version | Purpose | License | +| --- | --- | --- | --- | +| [CMake](http://cmake.org/) | >= 3.15 | for generating the build-system | BSD 3 | +| [pybind11](https://github.com/pybind/pybind11) | >= 2.13 | for building the Python-Wrapper | BSD 3 | +| [python](https://python.org/) | >= 3.7 | building and using the Python-Wrapper | PSFL |