Pyscript 1.3.0 release

The 1.3.0 release contains several cool new features and various bug fixes.

New features since 1.2.1 include:

• Multiple trigger decorators (@state_trigger, @time_trigger, @event_trigger or @mqtt_trigger) per function are now supported. See #157.
• Trigger decorators (@state_trigger, @time_trigger, @event_trigger or @mqtt_trigger) support an optional kwargs keyword argument that can be set to a dict of keywords and values, which are passed to the trigger function. See #157.
• The @service decorator now takes one or more optional string arguments to specify the name of the service, of the form "DOMAIN.SERVICE". The @service decorator can also be used multiple times as an alternative to using multiple arguments. The default continues to be pyscript.FUNC_NAME.
• Added @pyscript_executor decorator, which does same thing as @pyscript_compile and additionally wraps the resulting function with a call to task.executor. See #71.
• Added watch keyword argument to @state_trigger, which allows the list or set of state variables that are monitored to be specified. Normally that list is extracted from the @state_trigger expression.
• Errors in trigger-related decorators (eg, wrong arguments, unrecognized decorator type) raise exceptions rather than logging an error.
• Types of keyword arguments to trigger decorators are now checked and exceptions raised if incorrect.
• Added error checking for @pyscript_compile and @pyscript_executor to enforce there are no args or kwargs.
• Enabled GitHub discussions and added links to it; see #167.

Breaking changes since 1.2.1 include:

• Since decorator errors now raise exceptions, if you had a script with such an error that otherwise works, that script will now give an exception and fail to load. The error should be self-explanatory, and it's good to know so you can fix it.
• Since an exception is now raised if you call a function with an invalid keyword argument (ie, one not included in the function's argument definition), if you previously had a misspelled keyword argument that was silently ignored, you will now get an exception. It's good to know that is the case, and it should be easy to fix.
• The use of pyscript.config["apps"][YOUR_APP] to get application configuration is still available but now deprecated. The apps entry in pyscript.config will be removed in a future release. This is to prevent wayward applications from seeing configuration settings for other apps. The new pyscript.app_config variable should be used instead - it is set to pyscript.config["apps"][YOUR_APP] for each app.

Bug fixes since 1.2.1 include:

• Fixed @state_trigger with only a .old variable, which previously never triggered; reported by stigvi.
• Reload with global_ctx="*" and reload via Configuration -> Integrations UI menu now start triggers, reported by Fabio C.
• Fixed subscripts when running python 3.9.x.
• Timeouts that implement time triggers might infrequently occur a tiny time before the target time. A fix was added to do an additional short timeout when there is an early timeout, to make sure any time trigger occurs at or shortly after the target time (and never before).
• Fixes to locals(), exec() and eval() so that various scoping scenarios work correctly.
• When exception text is created, ensure lineno is inside code_list[]; with lambda function or eval it might not be.
• An exception is raised when a function is called with unexpected keyword parameters that don't have corresponding keyword arguments (however, the trigger parameter names are excluded from this check, since trigger functions are allowed to have any subset of keyword arguments).
• Non-local keyword arguments in inner functions are now correctly resolved to their outer scope; see #163
• Trigger on any state change now matches state names with leading digits; see #165

Enjoy!

Pyscript 1.2.1 release

The 1.2.1 release contains several bug fixes:

• Fixed @time_active so that ranges without dates (eg: plain sunrise, sunset or fixed times) work correctly on days after HASS was started. Reported by @markwhibbard; fix confirmed by @markwhibbard and @patrickfnielsen; see #149.
• lambda functions are now compiled so they are native python lambda functions that can be called from filter, map or anywhere that expects a regular python function. Reported by Greg sj3fk3.
• Raise an exception when a local variable is used before setting. Previously if a variable of the same name was defined globally, it was incorrectly used instead.
• Minor documentation fix regarding yaml file includes, submitted by @wsw70; see #153, #154.

Enjoy!

Pyscript 1.2.0 release

The 1.2.0 release contains some cool new features and some bug fixes.

New features since 1.1.0 include:

• Reload is automatically done whenever a script file, requirements.txt or yaml file below the pyscript folder is modified, created, renamed or deleted, or a directory is renamed, created or deleted; see #74. This uses the watchdog module which is already used by the folder_watcher integration.
• New functions task.create, task.current_task, task.cancel, task.name2id, task.wait, task.add_done_callback, task.remove_done_callback allow new background (async) tasks to be created, canceled, waited on, and completion callbacks to be added or deleted. Proposed by @dlashua and @valsr; see #112, #130, #143, #144.
• Added support for now to @time_trigger time specifications, which means the current date and time when the trigger was first evaluated (eg, at startup or when created as an inner function or closure), and remains fixed for the lifetime of the trigger. This allows time triggers of the form once(now + 5min) or period(now, 1hr). Proposed by @wsw70; see #139.
• Function decorators are now supported. However, the existing trigger decorators are still hardcoded (ie, not available as function calls), and decorators on classes are not yet supported. First implementation by @dlashua; see #43.
• New function decorator @pyscript_compile compiles a native Python function inside pyscript, which is helpful if you need a regular function (by default all pyscript functions are async coroutines) for task.executor, functions like map or filter, callbacks, or if you have code you want to run at compiled speed (see #71). The function body can't contain any pyscript-specific features, and closure of variables for an inner function that uses @pyscript.compile won't work either, since in pyscript local variables with scope binding are objects, not their native types. Proposed by @dlashua; see #71.
• A new variable pyscript.app_config is available in the global address space of an app's main file (ie, apps/YOUR_APP.py or apps/YOUR_APP/__init__.py) and is set to the YAML configuration for your app (ie, pyscript.config["apps"][YOUR_APP]). The latter is still available, but is deprecated and the apps entry in pyscript.config will be removed in a future release to prevent wayward applications from seeing configuration settings for other apps.
• Updated croniter to 1.0.2.
• Updated docs to explain how secret parameter values can be stored and retrieved from yaml configuration, by @exponentactivity; see #124.
• Report parsing errors on invalid @time_active arguments; by @dlashua; see #119.
• task.executor raises an exception when called with a pyscript function.

Breaking changes since 1.1.0 include:

None. However, the use of pyscript.config["apps"][YOUR_APP] to get application configuration is still available but now deprecated. The apps entry in pyscript.config will be removed in a future release. This is to prevent wayward applications from seeing configuration settings for other apps. The new pyscript.app_config variable should be used instead - it is set to pyscript.config["apps"][YOUR_APP] for each app.

Bug fixes since 1.1.0 include:

• Fixed shutdown trigger for case where it calls task.unique(); reported by @dlashua (#117).
• Duplicate @service function definitions (ie, with the same name) now correctly register the service, reported by @wsw70; see #121.
• Added error message for invalid @time_active argument, by @dlashua; see #118.
• The scripts subdirectory is now recursively traversed for requirements.txt files.
• Inner functions and classes (defined inside a function) are added to global symbol table if declared as global.
• Reload all scripts if global settings allow_all_imports or hass_is_global change; see #74.
• Methods bound to class instances use weakrefs so that __del__ works; reported by @dlashua; see #146.
• Pyscript user-defined functions (which are all async) can now be called from native python async code; see #137.
• Internals that call open() now set encoding=utf-8 so Windows platforms use the correct encoding; see #145.
• On Windows, python is missing locale.nl_langinfo, which caused startup to fail when the locale-specific days of week were extracted. Now the days of week in time trigger expressions are available on Windows, but only in English; see #145.
• task.name2id() raises NameError if task name is undefined. Also added kwargs to task.wait().
• Added "scripts/**" to REQUIREMENTS_PATHS, so deeper directories are searched.
• Fixed typos in task reaper code, by @dlashua; see #116.
• Fixed exception on invalid service call positional arguments, reported by @huonw; see #131.

Thanks to @dlashua for proposing, implementing and testing several new features in this release.

Enjoy!

Pyscript 1.1.0 release

The 1.1.0 release contains some cool new features and some bug fixes.

New features:

• New decorator @mqtt_trigger supports triggering off mqtt events, including an optional trigger expression, by @dlashua (#98, #105).
• All .py files below the pyscript/scripts directory are autoloaded, recursively, allowing you to arrange your scripts in sub-directories (or deeper) (#97).
• pyscript.reload only reloads changed files (changed contents, mtime, or an app's yaml configuration). All files in an app or module are reloaded if any one has changed, and any script, app or module that imports a changed modules (directly or indirectly) is also reloaded. Setting the optional global_ctx service parameter to * forces reloading all files (which is the behavior in 1.0.0 and earlier). See #106.
• Any script file name or directory name starting with # is skipped, which is an in-place way of disabling a specific script, app or directory tree; think of this as "commenting" the script or directory (#97).
• Added state_hold_false=None optional period in seconds to @state_trigger() and task.wait_until(). This requires the trigger expression to be False for at least that period (including 0) before a successful trigger. Setting this optional parameter makes state triggers edge triggered (ie, triggers only on transition from False to True), instead of the default level trigger (ie, only has to evaluate to True). Proposed by @tchef69 (#89), with suggestions from @dlashua (#95).
• @time_trigger now supports a "shutdown" trigger, which occurs on HASS shutdown or whenever the trigger function is no longer referenced (eg: during reload, or redefinition in Jupyter), proposed by @dlashua (#103).
• task.unique now allows multiple names to be "owned" by each task to support cases where you need to be exclusive against multiple other functions; provided by @dlashua.
• del and a new function state.delete() can delete state variables and state variable attributes.
• Updated versions in tests/requirements_test.txt.

Breaking changes:

• The pyscript.reload service only reloads changed files; prior behavior of reloading all files can be requested by setting the optional global_ctx service parameter to *.
• The trigger_time trigger function argument is now set to "startup" for a startup trigger, rather than None. This was done to be consistent with the new shutdown trigger, which calls the trigger function with trigger_time="shutdown". If you use trigger_time is None to detect a startup trigger, you'll need to change that to trigger_time == "startup" (see #103).

The bug fixes include:

• State setting now copies the attributes, to avoid a strange MappingProxyType recursion error inside HASS; reported by @github392 (#87).
• The deprecated function state.get_attr was missing an await, which causes an exception; it now works correctly, but please use state.getattr since the old function will be removed in a future release; reported and fixed by @dlashua (#88).
• The packaging module is installed if not found, since certain HASS configurations might not include it; fixed by @raman325 (#90, #91).

Thanks to @dlashua for adding @mqtt_trigger and multiple names in task.unique(), proposing several new features, as well as bug fixes and testing, and @raman325 for some bug fixes.

Enjoy!

Pyscript 1.0.0 release

The 1.0.0 release contains quite a few new features and some bug fixes. Given the number of new features, I decided to bump the version to 1.0.0 and switch to the more normal version number format.

New features

• Pyscript state variables (entity_ids) can be persisted across pyscript reloads and HASS restarts, from @swazrgb and @dlashua (#48).
• Entities domain.entity now support a virtual method service (eg, domain.entity.service()) that calls the service domain.service for any service that has an entity_id parameter, with that entity_id set to domain.entity. Proposed by @dlashua (#64).
• @state_trigger now supports triggering on an attribute change with "domain.entity.attr" and any attribute change with "domain.entity.*", from @dlashua (#82)
• State variables now support virtual attributes last_changed and last_updated for the UTC time when state values or any attribute was last changed.
• State variable attributes can be set by direct assignment, eg: DOMAIN.name.attr = value. An equivalent new function state.setattr() allows a specific attribute to be set.
• State variable values (eg, from domain.entity or state.get()) now include attributes that can be accessed after they are assigned to another, normal, variable.
• @state_trigger and task.wait_until now have an optional state_hold duration in seconds that requires the state trigger to remain true for that period of time. The trigger occurs after that time elapses. If the state trigger changes to false before the time expires, the process of waiting for a new trigger starts over.
• @time_active now has an optional hold_off duration in seconds, which ignores a new trigger if the last one happened within that time. Can be used for rate limiting or debouncing. Also, @time_active can now take zero time range arguments, in case you want to just specify hold_off.
• The hass object is available in all pyscript global contexts if the hass_is_global configuration parameter is true (default false). This allows access to HASS internals that might not be otherwise be exposed by pyscript. Use with caution (#51).
• Improvements to UI config flow, including allowing parameters to be updated, and the UI reload now works the same as the pyscript.reload service call, from @raman325 (#53)
• Added inbound context variable to trigger functions and support optional context setting on state, event firing and service calls. Proposal and PR from @dlashua (#50, #60).
• Logbook now supported using context and informational message based on trigger type. Proposal and PR from @dlashua (#50, #62).
• Required Python packages can be specified in requirements.txt files at the top-level pyscript directory, and each module's or app's directory. Those files are read and any missing packages are installed on HASS startup and pyscript reload. If a specific version of a package is needed, it can be pinned using the format package_name==version. Contributed by @raman325 (#66, #68, #69, #70, #78).
• The reload service now takes an optional parameter global_ctx that specifies just that global context is reloaded, eg: global_ctx="file.my_scripts". Proposed by @dlashua (#63).
• The state.get_attr() function has been renamed state.getattr(). The old function is still available and will be removed in some future release (it logs a warning when used).
• VSCode connections to pyscript's Jupyter kernel now work. Two changes were required: VSCode immediately closes the heartbeat port, which no longer causes pyscript to shut down the kernel. Also, stdout messages are flushed prior to sending the execute complete message. This is to ensure log and print messages get displayed in VSCode. One benign but unresolved bug with VSCode is that when you connect to the pyscript kernel, VSCode starts a second pyscript Jupyter kernel, before shutting that second one down.
• Service calls now accept blocking and limit parameters. The default behavior for a service call is to run it in the background, but using blocking=True will force a task to wait up to limit seconds for the service call to finish executing before continuing. Contributed by @raman325 (#85).

Breaking change due to bug fix

The @state_trigger expression is only evaluated when at least one of the state variables or attributes specifically mentioned in the expression have changed. This was fixed by @dlashua in #82, add see issue #73. Previously, any change to the state variable (eg, an attribute) would cause the @state_trigger expression to be evaluated, even if that attribute didn't appear in the expression. Although this is a bug fix, it does change when @state_trigger might cause a trigger. For example, if you use the trigger-on-any change form @state_trigger("domain.entity"), prior to 1.0.0 it would trigger on any state value change or any attribute change. In 1.0.0 and later, it will only trigger on a state value change. Also, if you used this form to make sure domain.entity had actually changed to new_value:

@state_trigger("domain.entity == 'new_value' and domain.entity.old != 'new_value'")

that will still work in 1.0.0, but confirming domain.entity.old is not new_value is no longer necessasry, and that 2nd clause can be safely removed.

Bug Fixes

• Jupyter autocomplete now works on multiline code blocks.
• Improved error message reporting for syntax errors inside f-strings.
• Fixed incorrect global context update on calling module that, in turn, does a callback (#58).
• task.wait_until no longer silently ignores unrecognized keyword arguments (#80).
• task.wait_until incorrectly ignored the keyword optional state_check_now argument (#81).

Thanks to @dlashua, @raman325 and @swazrgb for all their contributions to this release.

Enjoy!

Pyscript 0.32 release

The 0.32 release contains a couple of new features and one important bug fix.

New features:

• @state_trigger now supports just a state variable name string argument, which triggers on any change. This replaces the old confusing "True or domain.entity_id" form, although that still works.
• @state_trigger can now take multiple arguments, and any argument can be a list of string expressions or string state variable names (trigger on any change). All the conditions are logically ored together.
• Updates to the configuration setting allow_all_imports now take effect on reload.
• Improved error reporting during config flow.

The bug fix is to clean up a task to avoid a stall during HASS restart.

Thanks to @raman325 for supporting config changes on reload and improved error reporting in config flow.

Enjoy!

Pyscript 0.31 release

The 0.31 release contains one significant new feature and several bug fixes.

The new feature is config flow support, which allows setup of pyscript (including allow_all_imports) from the UI, while still allowing optional app configuration via yaml. Also, __name__ is now supported so modules, packages or apps can know their name.

On the infrastructure side, pylint is now included in the pre-commit, push and PR flows, and all tests now use pytest_homeassistant_custom_component instead of pytest_homeassistant.

Bug fixes include:

• global variable binding is done at run time
• class methods use the caller's execution context
• trigger actions now execute in the their own execution context
• task.unique() doesn't kill itself if the same task previously called it
• module and package functions use their own global symbol table, instead of caller's

Thanks to @raman325 for developing the config flow support, and to @dlashua for discovering and reporting all those bugs and providing concise failing examples.

Enjoy!

Pyscript 0.30 release

The 0.30 release contains a number of new features, and several bug fixes.

The main new features are:

• support for apps and importable modules and packages below the <config>/pyscript directory
• support for trigger functions as closures, allowing factory functions to create multiple similar trigger functions
• added a variable pyscript.config that allows access to yaml configuration settings
• added task.executor() for functions that block (eg, doing I/O) or have long running times
• added support for with, async for, lambda and named expressions
• use the croniter package for cron functions
• improved exception handling and reporting

Breaking changes:

• task.unique() now only applies within the current global context; the same name used in other global contexts will not be affected
• accessing an undefined state variable now throws a NameError exception, instead of returning None, except in @state_trigger or @state_active, where undefined state variables, attributes or the .old value will still evaluate to None if not defined

Infrastructure:

• moved documentation from README to hacs-pyscript.readthedocs.io and added more material
• added precommit checks for black, isort, flake8 and pytest
• set line-length to 109 for black/flake8/isort
• revamped pytest setup
• improved test coverage

Bug fixes include:

• variable scoping in function closures now works correctly
• fixed augmented assignments
• list/set/dict comprehension looping vars are now in implicitly nested scope
• del works correctly for non-locally scoped variables
• sunrise and sunset times now correct for dates other than today

Thanks to @fleXible for several PRs (adding croniter, fixing sunset/sunrise, revamping test infrastructure, refactoring and precommits, among others) and @dlashua for extensive testing, various bug reports and feature requests.

Enjoy!

Pyscript 0.21 release

The 0.21 release contains a number of new features, and several bug fixes.

The main new features are:

• language support for classes, list/dict/set comprehensions, and assert
• an optional configuration parameter allow_all_imports can be set to true to allow any package to be imported (thanks to @basnijholt, #8)
• "startup" is now a valid @time_trigger time_spec, which allows a function to be called on startup and at additional specified times (#7)
• @task_unique is a new decorator with the same arguments as task.unique() (#1)
• added state.names() to get a list of all entity ids for a given domain, or all if a domain is not specified (#12)
• added state.get_attr() to get all the attributes (in a dict) for a state variable
• added print() as an alias for log.debug(); it currently only supports a single argument
• autocomplete in Jupyter now includes Python keywords

One breaking change is that assigning to a state variable (HASS entity id) now preserves its existing attributes (previously, setting a state variable would delete its attributes). The state.set() function can be used to optionally overwrite or remove all attributes, or to set specific attributes while preserving others.

Bug fixes include:

• tuple assignments are fixed (fixes #14, #15)
• charset encoding on Jupyter messages is now utf-8 (fixes hass-puscript-jupyter #10)

Enjoy!

Pyscript 0.20 release

Pyscript 0.20 is a significant release that supports the Jupyter frontends (eg, notebook, console, lab). This allows fully interactive development and testing of pyscript functions, services, triggers and automation logic. Auto-completion with TAB is supported. See this README.md for more information.

To use Jupyter with pyscript you need to install a kernel shim from the hass-pyscript-jupyter repository.

Other new pyscript features include support for eval(), exec(), globals(), locals() and exceptions (try, except and raise). Error reporting and handling have been improved.

One small change that isn't backward compatible is that the logger path for pyscript has changed from homeassistant.components.pyscript to custom_components.pyscript. Also, the sub-path for pyscript functions has changed. To specify the logging level of a pyscript function FUNCNAME in a pyscript file FILE (without the .py extension), the logger path used in the logger configuration is now:

custom_components.pyscript.file.FILENAME.FUNCNAME

(In 0.11 it was homeassistant.components.pyscript.func.FUNCNAME, which is ambiguous if the same function name is used in different pyscript files.)

There are also a few bug fixes, including an issue that caused task.unique() to not terminate the specified task after the first time.

Enjoy!