automatically updated release notes for mega-20181014

[UX] Warn user for incomplete network IP settings

.gitignore

@@ -18,3 +18,9 @@ lib/readme.txt
 src/Custom.h
 /ESPEasy
 test/output_export.cpp
+.vscode
+.vscode/.browse.c_cpp.db*
+.vscode/c_cpp_properties.json
+.vscode/launch.json
+
+docs/build/

.travis.yml

@@ -22,9 +22,10 @@ addons:
 
 
 install:
-  - pip install -U platformio
+  - pip install -U platformio sphinx recommonmark sphinx_bootstrap_theme
 
 script:
+  # - bash ./preflight.sh # make sure input files are OK before wasting time with prereqs
   # - cppcheck --enable=warning src/*.ino -q --force -I src --include=src/ESPEasy.ino --error-exitcode=1
   # - ./memanalyzer.py ~/.platformio/packages/toolchain-xtensa/bin/xtensa-lx106-elf-objdump
   - PLATFORMIO_BUILD_FLAGS="-D CONTINUOUS_INTEGRATION" platformio run

README.md

@@ -1,9 +1,11 @@
+|Latest Nightly  | Build Status | Patreon | Ko-Fi | PayPal |
+|-------|-------|-------|-------|-------|
+| [![GitHub version](https://img.shields.io/github/release/letscontrolit/ESPEasy/all.svg)](https://github.com/letscontrolit/ESPEasy/releases/latest) | [![Build Status](https://travis-ci.org/letscontrolit/ESPEasy.svg?branch=mega)](https://travis-ci.org/letscontrolit/ESPEasy) | [![donate](https://img.shields.io/badge/donate-Patreon-blue.svg)](https://www.patreon.com/GrovkillenTDer) | [![donate](https://img.shields.io/badge/donate-KoFi-blue.svg)](https://ko-fi.com/grovkillentder) | [![donate](https://img.shields.io/badge/donate-PayPal-blue.svg)](https://www.paypal.me/espeasy) |
+
+For ways to *support* us, see [this announcement on the forum](https://www.letscontrolit.com/forum/viewtopic.php?f=14&t=5787), or have a look at the [Patreon](https://www.patreon.com/GrovkillenTDer), [Ko-Fi](https://ko-fi.com/grovkillentder) or [PayPal](https://www.paypal.me/espeasy) links above.
+
 # ESPEasy (development branch)
 
-Badge  | Info  |
--------|-------|
-[![Build Status](https://travis-ci.org/letscontrolit/ESPEasy.svg?branch=mega)](https://travis-ci.org/letscontrolit/ESPEasy) | build status |
-[![GitHub version](https://img.shields.io/github/release/letscontrolit/ESPEasy/all.svg)](https://github.com/letscontrolit/ESPEasy/releases/latest) | latest nightly |
 
 Introduction and wiki: https://www.letscontrolit.com/wiki/index.php/ESPEasy#Introduction
 
@@ -39,4 +41,6 @@ ESPEasy_mega-20180102_dev_ESP8285_1024.bin     | ESP8285 with 1Mb flash  | Stabl
 
 Details and discussion are on the Experimental forum: https://www.letscontrolit.com/forum/viewforum.php?f=18
 
+Automated builds of the (new) documentation can be found at [ESPEasy.readthedocs.io](https://espeasy.readthedocs.io/en/latest/)
+
 We're also on IRC: #ESPEasy @freenode

before_deploy

@@ -22,7 +22,8 @@ for ENV in \
   dev_ESP8266PUYA_1024_VCC\
   hard_SONOFF_POW\
   hard_Shelly_1\
-  esp32dev;\
+  esp32dev\
+  esp32test_1M8_partition;\
 do
   echo
   echo "### Deploying environment $ENV for version $VERSION"
@@ -34,10 +35,14 @@ done
 
 
 #create a source structure that is the same as the original ESPEasy project (and works with the howto on the wiki)
-rm -rf dist/Source 2>/dev/null
-mkdir dist/Source
+#rm -rf dist/Source 2>/dev/null
+rm -rf dist/Docs 2>/dev/null
+
+mkdir -p dist/Source
+#mkdir -p dist/Docs
 cp -r lib dist/Source/
 cp -r src dist/Source/
+#cp -r docs/build/ dist/Docs/
 cp platformio.ini dist/Source/
 
 cd dist

dist/README.txt

@@ -0,0 +1,61 @@
+  _____ ____  ____
+ | ____/ ___||  _ \ ___  __ _ ___ _   _
+ |  _| \___ \| |_) / _ \/ _` / __| | | |
+ | |___ ___) |  __/  __/ (_| \__ \ |_| |
+ |_____|____/|_|   \___|\__,_|___/\__, |
+                                  |___/
+
+
+This is the distribution folder for ESPeasy.
+In here you will find a flash tool to program the ESP module.
+
+Also a number of binary images is included.
+
+The filename is quite descriptive:
+  ESP_Easy_mega-<date>_<buildType>_<chip>_<memorySize>.bin
+
+Build type can be:  (differ in included plugins)
+- normal  => Only Stable plugins and controllers
+- test    => Stable + Testing
+- dev     => Stable + Testing + Development
+
+There is also a number of special builds:
+- normal_IR => "Normal" + IR receiver/transmitter plugins and library
+- hard_xxxxx => Special builds for some off-the-shelf hardware.
+
+Chip can be:
+- ESP8266      => Most likely option
+- ESP8285      => Used in some Sonoff modules
+- ESP32        => Experimental support at this moment
+- ESP8266PUYA  => For ESP modules with a flash chip labeled "PUYA" (often 1 MB)
+
+The "PUYA" version deserves a bit more attention.
+If you cannot save settings, better try to see if the flash chip has "PUYA" written on it.
+If so, try the "PUYA" build.
+
+MemorySize can be:
+- 1024  => 1 MB flash modules (e.g. almost all Sonoff modules)
+- 2048  => 2 MB flash modules (e.g. Shelly1/WROOM02)
+- 4096  => 4 MB flash modules (e.g. NodeMCU/ESP32)
+
+ESP32 now has 2 builds:
+- esp32dev   Using the default partition layout (1.4 MB for the sketch)
+- esp32test_1M8_partition   Larger sketch partition (1.8MB) smaller SPIFFS (316 kB)
+
+Please note that changing between those versions will destroy the settings!
+The SPIFFS partition will be lost and that contains all settings.
+
+
+To help recover from a bad flash, there are also blank images included.
+- blank_1MB.bin
+- blank_2MB.bin
+- blank_4MB.bin
+
+When the wrong image is flashed, or the module behaves unstable, or is in a reboot loop,
+flash these images first and then the right image for the module.
+
+Another great flash tool can be found here: (Windows only)
+  https://github.com/Grovkillen/ESP_Easy_Flasher
+
+For more information, see: https://github.com/letscontrolit/ESPEasy
+Or our forum: https://www.letscontrolit.com/forum/

dist/Release_notes.txt

@@ -1,3 +1,424 @@
+-------------------------------------------------
+Changes in release mega-20181014 (since mega-20181011)
+-------------------------------------------------
+
+Release date: Sun Oct 14 04:00:13 CEST 2018
+
+Gijs Noorlander (1):
+      [#1799] Add SSID to main page
+
+TD-er (5):
+      [#1891] Generic UPD controller - send only first value from sensor
+      [Controller] Fix send only first value
+      Time#Initialized event in rules before %syshour% is set
+      [UX] Warn user for incomplete network IP settings
+      [#1895] Incorrect number bytes read I2C_read32_reg
+
+
+-------------------------------------------------
+Changes in release mega-20181011 (since mega-20181010)
+-------------------------------------------------
+
+Release date: Thu Oct 11 04:00:24 CEST 2018
+
+TD-er (5):
+      [C009] Reduce memory usage of FHEM HTTP delay queue
+      [Controller] Set default timeout to 300 msec (was 100 msec)
+      Make sure to load ExtraTaskSettings when needed
+      [Travis Doc] Remove building documentation by Travis
+      [WebLog] Disable weblog when it is not being read
+
+
+-------------------------------------------------
+Changes in release mega-20181010 (since mega-20181009)
+-------------------------------------------------
+
+Release date: Wed Oct 10 04:00:09 CEST 2018
+
+Plebs (1):
+      Update src/_P019_PCF8574.ino
+
+
+-------------------------------------------------
+Changes in release mega-20181009 (since mega-20181008)
+-------------------------------------------------
+
+Release date: Tue Oct  9 04:00:18 CEST 2018
+
+TD-er (7):
+      [Docs] Add favicon to documentation
+      [Docs] Add Travis check for documentation and stop build for docs only
+      [Docs] Run Travic documentation check before compiling project
+      [Docs] Disable travis_terminate for now
+      [JSON] Replace \n and \r in text fields
+      [LWIP1.4] Move back to LWIP1.4
+      [StringConverter] Some optimisations to reduce memory allocation
+
+
+-------------------------------------------------
+Changes in release mega-20181008 (since mega-20181007)
+-------------------------------------------------
+
+Release date: Mon Oct  8 04:00:09 CEST 2018
+
+Gijs Noorlander (1):
+      Added link to ReadTheDocs
+
+Grovkillen (1):
+      [Sphinx] added extension "sphinx.ext.imgconverter"
+
+TD-er (3):
+      [Sphinx] Initial setup for Sphinx documentation
+      [Docs] First ESPEasy documentation (Controllers)
+      [Docs] Added some directory structure and template
+
+
+-------------------------------------------------
+Changes in release mega-20181007 (since mega-20181006)
+-------------------------------------------------
+
+Release date: Sun Oct  7 04:00:09 CEST 2018
+
+Max (2):
+      Force slash to be converted in htmlEscape
+      Fix XSS-Vulnerabilites on wifi configuration pages
+
+Plebs (3):
+      Add three new TOGGLE commands
+      fixed float to int conversion
+      added "mcpgpiotoggle" command
+
+TD-er (3):
+      [ESP32] Allow Wrover dev kit to be used for PIO debug
+      [Stack] Move Arduino 'cont' stack into user RAM
+      [SPIFFS] Make sure to close open files on SPIFFS
+
+
+-------------------------------------------------
+Changes in release mega-20181006 (since mega-20181004)
+-------------------------------------------------
+
+Release date: Sat Oct  6 04:00:13 CEST 2018
+
+TD-er (3):
+      [Stack] Increase stack to 5k and reduce stack allocations in rules
+      [WiFi] Add delay to connection attempts in ControllerSettings
+      [Build flags] Remove NO_EXTRA_4K_HEAP
+
+
+-------------------------------------------------
+Changes in release mega-20181004 (since mega-20181003)
+-------------------------------------------------
+
+Release date: Thu Oct  4 04:00:08 CEST 2018
+
+Bartlomiej Zimon (1):
+      - partial fix for #1723
+
+JojoS (1):
+      added error checking and logging
+
+JojoS62 (5):
+      added float calculation for time to distance conversion
+      added more settings: unit, filtertype and size
+      max distance depends on measurement unit
+      code clean up
+      fixed 'unused variable' warning
+
+TD-er (3):
+      [sendHttp] #1830 Set timeout and early exit on timeout reached
+      [WiFiClient] Set timeout and make it configurable for controllers
+      [Core 2.4.1] Move back to core 2.4.1 from 2.4.2
+
+Thomas (1):
+      Update _P075_Nextion.ino (#1)
+
+
+-------------------------------------------------
+Changes in release mega-20181003 (since mega-20181002)
+-------------------------------------------------
+
+Release date: Wed Oct  3 04:00:18 CEST 2018
+
+David Conran (2):
+      Acknowledge credit & origin of code used in IRTX
+      Acknowledge credit & origin of code used in _P016IR.ino
+
+
+-------------------------------------------------
+Changes in release mega-20181002 (since mega-20181001)
+-------------------------------------------------
+
+Release date: Tue Oct  2 04:00:29 CEST 2018
+
+Plebs (1):
+      split parseTemplate function
+
+TD-er (4):
+      [#1798] Bugfix Sonoff Si7021 stability
+      [ESP32] Add default partition table for reference
+      [ESP32] Update to PIO espressif32@1.4.0
+      [Stack] Report stack usage on ESP32 and increase stack for ESP8266normal
+
+
+-------------------------------------------------
+Changes in release mega-20181001 (since mega-20180930)
+-------------------------------------------------
+
+Release date: Mon Oct  1 13:13:13 CEST 2018
+
+TD-er (4):
+      [#1824] Show stack size on main page
+      [Heap] Use NO_EXTRA_4K_HEAP to get old stack address space
+      [Stack] Check if a block sent to save on SPIFFS is stack allocated
+      [Stack] Revert move Settings from stack to heap
+
+
+-------------------------------------------------
+Changes in release mega-20180930 (since mega-20180927)
+-------------------------------------------------
+
+Release date: Sun Sep 30 04:00:24 CEST 2018
+
+JojoS (5):
+      fixed distance reading
+      on INIT, delete sensordef before adding new
+      removed sensordefs helper structure
+      changed implementation to use shared_ptr instead of helper structure
+      minor change: unit was missing for threshold in 'state' mode
+
+TD-er (2):
+      [Default Settings] Properly clear settings when reset to default
+      [Defaults] Other default initializations
+
+
+-------------------------------------------------
+Changes in release mega-20180927 (since mega-20180924)
+-------------------------------------------------
+
+Release date: Thu Sep 27 04:00:17 CEST 2018
+
+TD-er (7):
+      [ESP32] Change partition size (will clear all settings) and add plugins
+      [Travis] Uninitialized variable in P073_7DGT
+      [Stack] Move Settings from stack to heap
+      [ESP32] RTOS move handle_schedule() to core1 and increase stack
+      [Info] Add board name to the sysinfo page.
+      [ESP32] Split new partition layout to separate build
+      [Build] Add esp32test_1M8_partition build and warning on ESP32 partition
+
+
+-------------------------------------------------
+Changes in release mega-20180924 (since mega-20180923)
+-------------------------------------------------
+
+Release date: Mon Sep 24 04:00:20 CEST 2018
+
+Benno Eigenmann (1):
+      Fix platformio
+
+TD-er (2):
+      [MQTT] Work around for lost MQTT connection
+      [ESP32] Fix I2C handling to allow I2C devices to work
+
+sakinit (1):
+      change to platform version core_esp32_1_3_0
+
+stefan (1):
+      Change type of sensor for MCP and PCF devices
+
+
+-------------------------------------------------
+Changes in release mega-20180923 (since mega-20180922)
+-------------------------------------------------
+
+Release date: Sun Sep 23 04:00:12 CEST 2018
+
+TD-er (2):
+      [MQTT import] Fix crash at boot when plugin is enabled
+      [BME280] Fix slow reading of BME280 causing watchdog resets
+
+sakinit (5):
+      ESP32 build issues fixed
+      Fix ESP32 stack overflow in addPinSelect when called from P1WifiGateway
+      ESP32: Split webserver spiffs filelist into pages
+      Reverted Webserver change
+      ESP32 platform updated to 1.2.0; Webserver replaced by platform one
+
+
+-------------------------------------------------
+Changes in release mega-20180922 (since mega-20180916)
+-------------------------------------------------
+
+Release date: Sat Sep 22 04:00:22 CEST 2018
+
+Gijs Noorlander (1):
+      Revert "Revert "Added 2 new operators in Calculate() function""
+
+TD-er (2):
+      Revert "[Scheduler] Disable command scheduler"
+      Revert "Revert "Fix elseif & Add if-elseif-else nesting levels""
+
+
+-------------------------------------------------
+Changes in release mega-20180916 (since mega-20180915)
+-------------------------------------------------
+
+Release date: Sun Sep 16 04:00:11 CEST 2018
+
+Gijs Noorlander (3):
+      Revert "Added 2 new operators in Calculate() function"
+      Revert "Fix elseif & Add if-elseif-else nesting levels"
+      Added warning to release notes 20180915
+
+TD-er (1):
+      [HW Watchdog] Backgroundtasks instead of yield during rules handling
+
+
+-------------------------------------------------
+Changes in release mega-20180915 (since mega-20180914)
+-------------------------------------------------
+
+Release date: Sat Sep 15 04:00:15 CEST 2018
+Known issue:
+      Causes reboots. (see [#1748](https://github.com/letscontrolit/ESPEasy/issues/1748) )
+
+
+Plebs (1):
+      Added 2 new operators in Calculate
+
+TD-er (4):
+      [Scheduler] Disable command scheduler
+      [Controller settings] Validate settings on load/save
+      [#1713] Make hostname-unitnumber optional & validate settings
+      [#1698] Incorrect sunrise value southern hemisphere
+
+svmac (4):
+      Fix elseif & Add if-elseif-else nesting levels
+      reenabled code about !
+      some code optimization
+      fix mistake
+
+
+-------------------------------------------------
+Changes in release mega-20180914 (since mega-20180910)
+-------------------------------------------------
+
+Release date: Fri Sep 14 04:00:17 CEST 2018
+
+Gijs Noorlander (1):
+      Added PayPal link
+
+giddyhup (1):
+      Fix IR plugin passing received code
+
+Plebs (7):
+      Use parameters inside rules when calling commands
+      few changes
+      fixed ESP32 compatibility issue
+      fixed calls to F-macro
+      adding more parameters to EVENT
+      final version with '=' sign as a prefix to identify the formula in rules
+      added %eventvalue4%
+
+TD-er (2):
+      [Flash tools] Add blank files and README.txt
+      [CrashRecover] Disable failing step based on failed boot count
+
+sakinit (1):
+      Feature: Split webserver spiffs filelist into pages
+
+
+-------------------------------------------------
+Changes in release mega-20180910 (since mega-20180909)
+-------------------------------------------------
+
+Release date: Mon Sep 10 04:00:22 CEST 2018
+
+TD-er (2):
+      [HTTP] Command SendToHttp cannot connect
+      [Domoticz IDX] Allow for IDX of > 9999
+
+sakinit (1):
+      Reduced buffer size to prevent stack overflow
+
+
+-------------------------------------------------
+Changes in release mega-20180909 (since mega-20180908)
+-------------------------------------------------
+
+Release date: Sun Sep  9 04:00:14 CEST 2018
+
+Gijs Noorlander (1):
+      Revert [WDT] Add watchdog feed to backgroundtasks()
+
+TD-er (5):
+      [#1720] WiFi read all lines from slow servers
+      Add forward declaration for void backgroundtasks()
+      [Serial Log] Limit serial log to prevent buffer overflow
+      [ESP32] Fix Build, ESP32 doesn't have the availableForWrite
+      [Serial log] Use buffer to prevent Serial buffer to overflow
+
+
+-------------------------------------------------
+Changes in release mega-20180908 (since mega-20180904)
+-------------------------------------------------
+
+Release date: Sat Sep  8 04:00:20 CEST 2018
+
+Gijs Noorlander (1):
+      Update README.md
+
+TD-er (37):
+      [controller delay] Let controllers handle their own delay and buffer
+      [MQTT] Add controller delay queue for MQTT
+      [Controller] Make ControllerDelayHandler templated and comment
+      [C001] Let C001 controller use queue
+      [C003] Let C003 controller use queue
+      [C004] Let C004 controller use queue and simplify using macros
+      Added `USES_Cxxx` checks
+      [C007] Let C007 controller use queue
+      [MQTT] Use set controller settings on MQTT controllers
+      [Webserver] Add fields for new Controller settings
+      [C008] Let C008 use queue and some bugfixes
+      [#1666] Moved code duplication for HTTP requests to special functions
+      [C009] Removed port number from HTTP host header
+      [C009] Controller C009 now also uses Delay Queue
+      [Client Connect] Check for success on connecting with client.
+      [C009] Remove 'no port' option in HTTP packet
+      [HTTP Auth] Now using proper index for retrieving credentials
+      [HTTP] Add user agent in request header
+      [C010] Controller C010 now also uses Delay Queue
+      [Cleanup] Merge code duplication between C008 and C010
+      [C011] Controller C011 now also uses Delay Queue
+      [Controller] Moved the 'magic' default and max values to defines
+      [C012] Controller C012 now also uses Delay Queue
+      [C013] Remove some obsolete function
+      [Nodes] Make Nodes list a map with dynamic size
+      [Nodes] Highlight current node in the list of nodes
+      [Cleanup] Memory optimisation in available controllers and plugins
+      [Nodes] Fix String copy to properly read node names
+      [MQTT] Give MQTT state in log when connect state changes.
+      [SPIFFS] Add SPIFFS size + usage to sysinfo page
+      [Watchdog] Add watchdog feed to backgroundtasks() function
+      [MQTT] Force MQTT client reconnect when wifi reconnects
+      [WDT] Add yield() and runbackgroundtasks() when parsing rules
+      [#1707] Getting logs from the /log even when password is not entered
+      [#1714] Parse notification sender to allow some sender formatting
+      [Scheduler] Do not schedule task device timer for tasks without interval
+      [Queue] Add memory check to determine full queue
+
+
+-------------------------------------------------
+Changes in release mega-20180904 (since mega-20180903)
+-------------------------------------------------
+
+Release date: Tue Sep  4 04:00:34 CEST 2018
+
+TD-er (1):
+      [#1702] Check for duplicate names bug
+
+
 -------------------------------------------------
 Changes in release mega-20180903 (since mega-20180826)
 -------------------------------------------------

docs/Makefile

@@ -0,0 +1,19 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line.
+SPHINXOPTS    =
+SPHINXBUILD   = sphinx-build
+SOURCEDIR     = source
+BUILDDIR      = build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/make.bat

@@ -0,0 +1,35 @@
+@ECHO OFF
+
+pushd %~dp0
+
+REM Command file for Sphinx documentation
+
+if "%SPHINXBUILD%" == "" (
+	set SPHINXBUILD=sphinx-build
+)
+set SOURCEDIR=source
+set BUILDDIR=build
+
+if "%1" == "" goto help
+
+%SPHINXBUILD% >NUL 2>NUL
+if errorlevel 9009 (
+	echo.
+	echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
+	echo.installed, then set the SPHINXBUILD environment variable to point
+	echo.to the full path of the 'sphinx-build' executable. Alternatively you
+	echo.may add the Sphinx directory to PATH.
+	echo.
+	echo.If you don't have Sphinx installed, grab it from
+	echo.http://sphinx-doc.org/
+	exit /b 1
+)
+
+%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%
+goto end
+
+:help
+%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%
+
+:end
+popd

docs/requirements.txt

@@ -0,0 +1,3 @@
+recommonmark
+Sphinx
+sphinx-bootstrap-theme

docs/source/Controller/Controller.rst

@@ -0,0 +1,66 @@
+Controller
+**********
+
+A controller is a component to enable a plugin to send data elsewhere.
+
+- Up-to 3 controllers can be active in ESPEasy.
+- Per plugin up-to 3 active controllers can be selected.
+- For some controllers an additional parameter can be given.
+
+For example, Domoticz needs an 'IDX' value to identify the configured entry in
+Domoticz for which new data is sent.
+
+Controller Parameters
+=====================
+
+Generic fields
+--------------
+
+- **Protocol** - The type of controller (e.g. ThingSpeak/OpenHAB MQTT/etc.)
+- **Locate Controller** - Selection between hostname/IP
+- **Controller Hostname/IP**  - The address to reach the selected service
+- **Controller Port** - TCP/UDP Port number (0...65536)
+- **Enabled** - Whether or not the controller is active.
+
+Send queue parameters
+---------------------
+
+Controllers have a queue to keep unsent messages.
+This queue is used to handle message bursts and also store messages which are recorded
+before WiFi connection is made or during lost connection.
+
+- **Minimum Send Interval** - Minimum time between two messages in msec.
+- **Max Queue Depth** - Maximum length of the buffer queue to keep unsent messages.
+- **Max Retries** - Maximum number of retries to send a message.
+- **Full Queue Action** - How to handle when queue is full, ignore new or delete oldest message.
+- **Client Timeout** - Timeout in msec for an network connection used by the controller.
+- **Check Reply** - When set to false, a sent message is considered always successful.
+
+
+Sample ThingSpeak configuration
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Some controllers, like ThingSpeak, need a specific configuration.
+ThingSpeak only allows a message every 15 seconds for the free accounts.
+
+- **Minimum Send Interval** - 15000 msec
+- **Max Queue Depth** - 1 (only report the last value)
+- **Max Retries** - 2
+- **Full Queue Action** - Delete Oldest
+- **Client Timeout** - 500 msec (server is online, so timeout must be a bit longer)
+
+
+Controller user credentials
+---------------------------
+
+- **Controller User** - User name (optional)
+- **Controller Password** - Password (optional)
+
+MQTT related settings
+---------------------
+
+- **Controller Subscribe** - Subscribe to the given topic.
+- **Controller Publish** - Publish to the given topic.
+- **Controller lwl topic** - Topic to which LWT (Last Will Testament) messages should be sent.
+- **LWT Connect Message** - Connection established message.
+- **LWT Disconnect Message** - Connection lost message (sent to broker during connect and published by broker when connection is lost)

docs/source/ESPEasy/AboutUs.rst

@@ -0,0 +1,2 @@
+About Us
+********

docs/source/ESPEasy/FunctionBlocks.rst

@@ -0,0 +1,2 @@
+ESPEasy Function Blocks
+***********************

docs/source/Participate/Documentation.rst

@@ -0,0 +1,35 @@
+Documentation
+*************
+
+For documentation we use Sphinx and this will be read by ReadTheDocs.
+We use the `Sphinx Bootstrap Theme <https://github.com/ryan-roemer/sphinx-bootstrap-theme>`_
+
+This documentation is included in the GitHub repository.
+It allows us to create documentation per version of ESPEasy.
+
+See also the `ESPEasy wiki <https://www.letscontrolit.com/wiki/index.php/ESPEasy>`_
+for more documentation which has not been moved here.
+
+Needed Python packages::
+
+   pip install sphinx recommonmark sphinx_bootstrap_theme
+
+PlatformIO with Atom
+====================
+
+With the ESPEasy project open in Atom, open the PIO terminal in Atom.
+
+Install dependencies::
+
+   cd docs
+   pip install ... (see needed Python packages above)
+
+Build on Windows::
+
+   cd docs
+   .\make.bat html
+
+Build on Linux/Mac::
+
+   cd docs
+   ./make html

docs/source/Plugin/Plugin.rst

@@ -0,0 +1,2 @@
+Plugins
+*******

docs/source/Reference/Command.rst

@@ -0,0 +1,2 @@
+Command Reference
+*****************

docs/source/Reference/SystemVariable.rst

@@ -0,0 +1,2 @@
+System Variables
+****************

docs/source/Rules/Rules.rst

@@ -0,0 +1,2 @@
+Rules
+*****

docs/source/conf.py

@@ -0,0 +1,286 @@
+# -*- coding: utf-8 -*-
+#
+# Configuration file for the Sphinx documentation builder.
+#
+# This file does only contain a selection of the most common options. For a
+# full list see the documentation:
+# http://www.sphinx-doc.org/en/master/config
+
+# -- Path setup --------------------------------------------------------------
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#
+# import os
+# import sys
+# sys.path.insert(0, os.path.abspath('.'))
+from recommonmark.parser import CommonMarkParser
+import sphinx_bootstrap_theme
+
+
+# -- Project information -----------------------------------------------------
+
+project = u'ESPEasy'
+copyright = u'2018, ESPEasy'
+author = u'Many'
+
+# The short X.Y version
+version = u''
+# The full version, including alpha/beta/rc tags
+release = u'2.1b1'
+
+
+# -- General configuration ---------------------------------------------------
+
+# If your documentation needs a minimal Sphinx version, state it here.
+#
+# needs_sphinx = '1.0'
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = [
+    'sphinx.ext.autodoc',
+    'sphinx.ext.intersphinx',
+    'sphinx.ext.todo',
+    'sphinx.ext.imgmath',
+    'sphinx.ext.imgconverter',
+]
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+source_parsers = {
+    '.md': CommonMarkParser,
+}
+
+# The suffix(es) of source filenames.
+# You can specify multiple suffix as a list of string:
+#
+source_suffix = ['.rst', '.md']
+#source_suffix = '.rst'
+
+# The master toctree document.
+master_doc = 'index'
+
+# The language for content autogenerated by Sphinx. Refer to documentation
+# for a list of supported languages.
+#
+# This is also used if you do content translation via gettext catalogs.
+# Usually you set "language" from the command line for these cases.
+language = None
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This pattern also affects html_static_path and html_extra_path.
+exclude_patterns = []
+
+# The name of the Pygments (syntax highlighting) style to use.
+pygments_style = None
+
+
+# -- Options for HTML output -------------------------------------------------
+html_logo = 'ESPEasy/espeasy_logo.png'
+html_favicon = 'ESPEasy/favicon.ico'
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+#
+#html_theme = 'sphinx_rtd_theme'
+html_theme = 'bootstrap'
+html_theme_path = sphinx_bootstrap_theme.get_html_theme_path()
+
+# Theme options are theme-specific and customize the look and feel of a theme
+# further.  For a list of options available for each theme, see the
+# documentation.
+#
+# html_theme_options = {}
+#html_theme_options = {
+#    'canonical_url': '',
+#    'analytics_id': '',
+#    'logo_only': False,
+#    'display_version': True,
+#    'prev_next_buttons_location': 'bottom',
+#    'style_external_links': False,
+#    # Toc options
+#    'collapse_navigation': True,
+#    'sticky_navigation': True,
+#    'navigation_depth': 4,
+#    'includehidden': True,
+#    'titles_only': False
+#}
+
+html_theme_options = {
+    # Navigation bar title. (Default: ``project`` value)
+    'navbar_title': "ESPEasy",
+
+    # Tab name for entire site. (Default: "Site")
+    'navbar_site_name': "Site",
+
+    # A list of tuples containing pages or urls to link to.
+    # Valid tuples should be in the following forms:
+    #    (name, page)                 # a link to a page
+    #    (name, "/aa/bb", 1)          # a link to an arbitrary relative url
+    #    (name, "http://example.com", True) # arbitrary absolute url
+    # Note the "1" or "True" value above as the third argument to indicate
+    # an arbitrary url.
+    'navbar_links': [
+        ("Rules", "rules"),
+        ("Forum", "https://www.letscontrolit.com/forum/viewforum.php?f=1", True),
+    ],
+
+    # Render the next and previous page links in navbar. (Default: true)
+    'navbar_sidebarrel': True,
+
+    # Render the current pages TOC in the navbar. (Default: true)
+    'navbar_pagenav': True,
+
+    # Tab name for the current pages TOC. (Default: "Page")
+    'navbar_pagenav_name': "Page",
+
+    # Global TOC depth for "site" navbar tab. (Default: 1)
+    # Switching to -1 shows all levels.
+    'globaltoc_depth': 2,
+
+    # Include hidden TOCs in Site navbar?
+    #
+    # Note: If this is "false", you cannot have mixed ``:hidden:`` and
+    # non-hidden ``toctree`` directives in the same page, or else the build
+    # will break.
+    #
+    # Values: "true" (default) or "false"
+    'globaltoc_includehidden': "true",
+
+    # HTML navbar class (Default: "navbar") to attach to <div> element.
+    # For black navbar, do "navbar navbar-inverse"
+    'navbar_class': "navbar navbar-inverse",
+
+    # Fix navigation bar to top of page?
+    # Values: "true" (default) or "false"
+    'navbar_fixed_top': "true",
+
+    # Location of link to source.
+    # Options are "nav" (default), "footer" or anything else to exclude.
+    'source_link_position': "none",
+
+    # Bootswatch (http://bootswatch.com/) theme.
+    #
+    # Options are nothing (default) or the name of a valid theme
+    # such as "cosmo" or "sandstone".
+    #
+    # The set of valid themes depend on the version of Bootstrap
+    # that's used (the next config option).
+    #
+    # Currently, the supported themes are:
+    # - Bootstrap 2: https://bootswatch.com/2
+    # - Bootstrap 3: https://bootswatch.com/3
+    'bootswatch_theme': "cerulean",
+
+    # Choose Bootstrap version.
+    # Values: "3" (default) or "2" (in quotes)
+    'bootstrap_version': "3",
+}
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['_static']
+
+# Custom sidebar templates, must be a dictionary that maps document names
+# to template names.
+#
+# The default sidebars (for documents that don't match any pattern) are
+# defined by theme itself.  Builtin themes are using these templates by
+# default: ``['localtoc.html', 'relations.html', 'sourcelink.html',
+# 'searchbox.html']``.
+#
+# html_sidebars = {}
+
+
+# -- Options for HTMLHelp output ---------------------------------------------
+
+# Output file base name for HTML help builder.
+htmlhelp_basename = 'ESPEasydoc'
+
+
+# -- Options for LaTeX output ------------------------------------------------
+
+latex_elements = {
+    # The paper size ('letterpaper' or 'a4paper').
+    #
+    # 'papersize': 'letterpaper',
+
+    # The font size ('10pt', '11pt' or '12pt').
+    #
+    # 'pointsize': '10pt',
+
+    # Additional stuff for the LaTeX preamble.
+    #
+    # 'preamble': '',
+
+    # Latex figure (float) alignment
+    #
+    # 'figure_align': 'htbp',
+}
+
+# Grouping the document tree into LaTeX files. List of tuples
+# (source start file, target name, title,
+#  author, documentclass [howto, manual, or own class]).
+latex_documents = [
+    (master_doc, 'ESPEasy.tex', u'ESPEasy Documentation',
+     u'Many', 'manual'),
+]
+
+
+# -- Options for manual page output ------------------------------------------
+
+# One entry per manual page. List of tuples
+# (source start file, name, description, authors, manual section).
+man_pages = [
+    (master_doc, 'espeasy', u'ESPEasy Documentation',
+     [author], 1)
+]
+
+
+# -- Options for Texinfo output ----------------------------------------------
+
+# Grouping the document tree into Texinfo files. List of tuples
+# (source start file, target name, title, author,
+#  dir menu entry, description, category)
+texinfo_documents = [
+    (master_doc, 'ESPEasy', u'ESPEasy Documentation',
+     author, 'ESPEasy', 'ESP82xx/ESP32 firmware to connect people with any level of experience to system automation.',
+     'Miscellaneous'),
+]
+
+
+# -- Options for Epub output -------------------------------------------------
+
+# Bibliographic Dublin Core info.
+epub_title = project
+
+# The unique identifier of the text. This can be a ISBN number
+# or the project homepage.
+#
+# epub_identifier = ''
+
+# A unique identification for the text.
+#
+# epub_uid = ''
+
+# A list of files that should not be packed into the epub file.
+epub_exclude_files = ['search.html']
+
+
+# -- Extension configuration -------------------------------------------------
+
+# -- Options for intersphinx extension ---------------------------------------
+
+# Example configuration for intersphinx: refer to the Python standard library.
+intersphinx_mapping = {'https://docs.python.org/': None}
+
+# -- Options for todo extension ----------------------------------------------
+
+# If true, `todo` and `todoList` produce output, else they produce nothing.
+todo_include_todos = True

docs/source/index.rst

@@ -0,0 +1,38 @@
+.. ESPEasy documentation master file, created by
+   sphinx-quickstart on Sat Oct 06 22:34:34 2018.
+   You can adapt this file completely to your liking, but it should at least
+   contain the root `toctree` directive.
+
+Welcome to ESPEasy's documentation!
+===================================
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Table of Contents
+
+   ESPEasy/AboutUs.rst
+   ESPEasy/FunctionBlocks.rst
+   Controller/Controller.rst
+   Plugin/Plugin.rst
+   Rules/Rules.rst
+
+.. raw:: latex
+
+   \appendix
+
+
+.. toctree::
+   :caption: Appendix
+
+   Participate/Documentation.rst
+   Reference/Command.rst
+   Reference/SystemVariable.rst
+
+
+
+Indices and tables
+==================
+
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`

esp32_partition_app1310k_spiffs1503k.csv

@@ -0,0 +1,7 @@
+# Name,   Type, SubType, Offset,  Size, Flags
+nvs,      data, nvs,     0x9000,  0x5000,
+otadata,  data, ota,     0xe000,  0x2000,
+app0,     app,  ota_0,   0x10000, 0x140000,
+app1,     app,  ota_1,   0x150000,0x140000,
+eeprom,   data, 0x99,    0x290000,0x1000,
+spiffs,   data, spiffs,  0x291000,0x16F000,

esp32_partition_app1810k_spiffs316k.csv

@@ -0,0 +1,7 @@
+# Name,   Type, SubType, Offset,  Size, Flags
+nvs,      data, nvs,     0x9000,  0x5000,
+otadata,  data, ota,     0xe000,  0x2000,
+app0,     app,  ota_0,   0x10000, 0x1D0000,
+app1,     app,  ota_1,   0x1E0000,0x1D0000,
+eeprom,   data, 0x99,    0x3B0000,0x1000,
+spiffs,   data, spiffs,  0x3B1000,0x4F000,

lib/NewPingESP8266/NewPingESP8266.cpp

@@ -6,12 +6,13 @@
 // ---------------------------------------------------------------------------
 
 #include "NewPingESP8266.h"
-
+//#include "ESPEasy-Globals.h"
 // ---------------------------------------------------------------------------
 // NewPingESP8266 constructor
 // ---------------------------------------------------------------------------
 
 NewPingESP8266::NewPingESP8266(uint32_t trigger_pin, uint32_t echo_pin, unsigned int max_cm_distance) {
+	_errorState = STATUS_SENSOR_READY;
 #if DO_BITWISE == true
 	_triggerBit = digitalPinToBitMask(trigger_pin); // Get the port register bitmask for the trigger pin.
 	_echoBit = digitalPinToBitMask(echo_pin);       // Get the port register bitmask for the echo pin.
@@ -58,7 +59,10 @@ unsigned int NewPingESP8266::ping(unsigned int max_cm_distance) {
 		while (!digitalRead(_echoPin))                // Wait for the ping echo.
 	#endif
 	  {
-			if (micros() > _max_time) return NO_ECHO; // Stop the loop and return NO_ECHO (false) if we're beyond the set maximum distance.
+			if ((_max_cm_distance > 0) && (micros() > _max_time)) {
+				_errorState = STATUS_MAX_DISTANCE_EXCEEDED;
+				return NO_ECHO; // Stop the loop and return NO_ECHO (false) if we're beyond the set maximum distance.
+			}
 	  }
 #else
 	#if DO_BITWISE == true
@@ -67,10 +71,14 @@ unsigned int NewPingESP8266::ping(unsigned int max_cm_distance) {
 		while (digitalRead(_echoPin))                 // Wait for the ping echo.
 	#endif
 		{
-			if (micros() > _max_time) return NO_ECHO; // Stop the loop and return NO_ECHO (false) if we're beyond the set maximum distance.
+			if ((_max_cm_distance > 0) && (micros() > _max_time)) {
+				_errorState = STATUS_MAX_DISTANCE_EXCEEDED;
+				return NO_ECHO; // Stop the loop and return NO_ECHO (false) if we're beyond the set maximum distance.
+			}
 		}
 #endif
 
+	_errorState = STATUS_MEASUREMENT_VALID;
 	return (micros() - (_max_time - _maxEchoTime) - PING_OVERHEAD); // Calculate ping time, include overhead.
 }
 
@@ -144,15 +152,39 @@ boolean NewPingESP8266::ping_trigger() {
 	#endif
 
 	#if URM37_ENABLED == true
-		if (!(*_echoInput & _echoBit)) return false;            // Previous ping hasn't finished, abort.
+		if (!(*_echoInput & _echoBit)) {								      	// Previous ping hasn't finished, abort.{
+			_errorState = STATUS_ECHO_STATE_ERROR;
+			return false;
+		}
 		_max_time = micros() + _maxEchoTime + MAX_SENSOR_DELAY; // Maximum time we'll wait for ping to start (most sensors are <450uS, the SRF06 can take up to 34,300uS!)
 		while (*_echoInput & _echoBit)                          // Wait for ping to start.
-			if (micros() > _max_time) return false;             // Took too long to start, abort.
+		{
+			if (millis() > (start + 50)) {
+				_errorState = STATUS_ECHO_START_TIMEOUT_50ms;
+				return false;
+			}
+			if ((_max_cm_distance > 0) && (micros() > _max_time)) {	// check max. distance if != 0 cm
+				_errorState = STATUS_ECHO_START_TIMEOUT_DISTANCE;
+				return false;             													// echo too late, max distance exceeded
+			}
+		}
 	#else
-		if (*_echoInput & _echoBit) return false;               // Previous ping hasn't finished, abort.
+		if (*_echoInput & _echoBit) { 				              		// Previous ping hasn't finished, abort.
+			_errorState = STATUS_ECHO_STATE_ERROR;
+			return false;
+		}
 		_max_time = micros() + _maxEchoTime + MAX_SENSOR_DELAY; // Maximum time we'll wait for ping to start (most sensors are <450uS, the SRF06 can take up to 34,300uS!)
 		while (!(*_echoInput & _echoBit))                       // Wait for ping to start.
-			if (micros() > _max_time) return false;             // Took too long to start, abort.
+		{
+			if (millis() > (start + 50)) {
+				_errorState = STATUS_ECHO_START_TIMEOUT_50ms;
+				return false;
+			}
+			if ((_max_cm_distance > 0) && (micros() > _max_time)) {	// check max. distance if != 0 cm
+				_errorState = STATUS_ECHO_START_TIMEOUT_DISTANCE;
+				return false;             													// echo too late, max distance exceeded
+			}
+		}
 	#endif
 #else
 	#if ONE_PIN_ENABLED == true
@@ -170,30 +202,51 @@ boolean NewPingESP8266::ping_trigger() {
 	#endif
 
 	#if URM37_ENABLED == true
-		if (!digitalRead(_echoPin)) return false;               // Previous ping hasn't finished, abort.
+		if (!digitalRead(_echoPin)) {
+			_errorState = STATUS_ECHO_STATE_ERROR;
+			return false;                // Previous ping hasn't finished, abort.
+		}
 		_max_time = micros() + _maxEchoTime + MAX_SENSOR_DELAY; // Maximum time we'll wait for ping to start (most sensors are <450uS, the SRF06 can take up to 34,300uS!)
 		while (digitalRead(_echoPin))                           // Wait for ping to start.
 		{
-			if (millis() > (start + 50)) return false;
-			if (micros() > _max_time) return false;             // Took too long to start, abort.
+			if (millis() > (start + 50)) {
+				_errorState = STATUS_ECHO_START_TIMEOUT_50ms;
+				return false;
+			}
+			if ((_max_cm_distance > 0) && (micros() > _max_time)) {	// check max. distance if != 0 cm
+				_errorState = STATUS_ECHO_START_TIMEOUT_DISTANCE;
+				return false;             													// echo too late, max distance exceeded
+			}
 		}
 	#else
-		if (digitalRead(_echoPin)) return false;                // Previous ping hasn't finished, abort.
+		if (digitalRead(_echoPin)) {
+			_errorState = STATUS_ECHO_STATE_ERROR;
+			return false;                // Previous ping hasn't finished, abort.
+		}
 		_max_time = micros() + _maxEchoTime + MAX_SENSOR_DELAY; // Maximum time we'll wait for ping to start (most sensors are <450uS, the SRF06 can take up to 34,300uS!)
 		while (!digitalRead(_echoPin))                          // Wait for ping to start.
 		{
-			if (millis() > (start + 50)) return false;
-			if (micros() > _max_time) return false;             // Took too long to start, abort.
+			if (millis() > (start + 50)) {
+				_errorState = STATUS_ECHO_START_TIMEOUT_50ms;
+				return false;
+			}
+			if ((_max_cm_distance > 0) && (micros() > _max_time)) {	// check max. distance if != 0 cm
+				_errorState = STATUS_ECHO_START_TIMEOUT_DISTANCE;
+				return false;             													// echo too late, max distance exceeded
+			}
   	}
 	#endif
 #endif
 
 	_max_time = micros() + _maxEchoTime; // Ping started, set the time-out.
+	_errorState = STATUS_ECHO_TRIGGERED;
+
 	return true;                         // Ping started successfully.
 }
 
 
 void NewPingESP8266::set_max_distance(unsigned int max_cm_distance) {
+	_max_cm_distance = max_cm_distance;
 #if ROUNDING_ENABLED == false
 	_maxEchoTime = min(max_cm_distance + 1, (unsigned int) MAX_SENSOR_DISTANCE + 1) * US_ROUNDTRIP_CM; // Calculate the maximum distance in uS (no rounding).
 #else
@@ -222,3 +275,12 @@ unsigned int NewPingESP8266::convert_in(unsigned int echoTime) {
 	return NewPingESP8266Convert(echoTime, US_ROUNDTRIP_IN); // Convert uS to inches.
 #endif
 }
+
+float NewPingESP8266::convert_cm_F(unsigned int echoTime) {
+	return ((float)echoTime / US_ROUNDTRIP_CM);              // Convert uS to centimeters (no rounding).
+}
+
+
+float NewPingESP8266::convert_in_F(unsigned int echoTime) {
+	return ((float)echoTime / US_ROUNDTRIP_IN);              // Convert uS to inches (no rounding).
+}

lib/NewPingESP8266/NewPingESP8266.h

@@ -194,15 +194,29 @@
 
 class NewPingESP8266 {
 	public:
+		enum errorState {
+			STATUS_SENSOR_READY,  								// no error, no measruement started
+			STATUS_MEASUREMENT_VALID, 						// no errors
+			STATUS_ECHO_TRIGGERED,								// echo triggered, intermediate state
+			STATUS_MAX_DISTANCE_EXCEEDED, 				// echo too late, maximum distance exceeded
+			STATUS_ECHO_START_TIMEOUT_50ms, 			// no echo start signal after 50 ms
+			STATUS_ECHO_START_TIMEOUT_DISTANCE,		// no echo start signal after max. distance
+			STATUS_ECHO_STATE_ERROR 				 			// echopin level not low after triggered
+		};
+
 		NewPingESP8266(uint32_t trigger_pin, uint32_t echo_pin, unsigned int max_cm_distance = MAX_SENSOR_DISTANCE);
+		~NewPingESP8266() { };
 		unsigned int ping(unsigned int max_cm_distance = 0);
 		unsigned long ping_cm(unsigned int max_cm_distance = 0);
 		unsigned long ping_in(unsigned int max_cm_distance = 0);
 		unsigned long ping_median(uint32_t it = 5, unsigned int max_cm_distance = 0);
 		static unsigned int convert_cm(unsigned int echoTime);
 		static unsigned int convert_in(unsigned int echoTime);
+		static float convert_cm_F(unsigned int echoTime);
+		static float convert_in_F(unsigned int echoTime);
 
-		unsigned int getMaxEchoTime() { return _maxEchoTime; }
+		unsigned int getMaxEchoTime() { return _maxEchoTime; };
+		errorState getErrorState() { return _errorState; };
 	private:
 		boolean ping_trigger();
 		void set_max_distance(unsigned int max_cm_distance);
@@ -218,6 +232,8 @@ class NewPingESP8266 {
 #endif
 		unsigned int _maxEchoTime;
 		unsigned long _max_time;
+		errorState _errorState;
+		unsigned int _max_cm_distance;
 };
 
 

platformio.ini

@@ -65,6 +65,7 @@ platform                  = https://github.com/platformio/platform-espressif8266
 
 [core_esp32_0_12_0]
 platform                  = espressif32@0.12.0
+lib_deps                  = "ESP32WebServer"
 
 [core_esp32_1_2_0]
 platform                  = espressif32@1.2.0
@@ -72,20 +73,23 @@ platform                  = espressif32@1.2.0
 [core_esp32_1_3_0]
 platform                  = espressif32@1.3.0
 
+[core_esp32_1_4_0]
+platform                  = espressif32@1.4.0
+
 [core_esp32_stage]
 platform                  = https://github.com/platformio/platform-espressif32.git#feature/stage
 
 [core_esp32]
-platform                  = ${core_esp32_0_12_0.platform}
+platform                  = ${core_esp32_1_4_0.platform}
 build_unflags             = -Wall
 build_flags               = -D BUILD_GIT='"${sysenv.TRAVIS_TAG}"'
-                            -lstdc++ -lsupc++
 lib_ignore                = AS_BH1750, ESP8266WiFi, ESP8266Ping, ESP8266WebServer, ESP8266HTTPUpdateServer, ESP8266mDNS, IRremoteESP8266, ESPEasy_ESP8266Ping, SerialSensors, SDM
-lib_deps                  = ESP32WebServer
+lib_deps                  = ""
 monitor_speed             = 115200
 
 
 
+
 ; Build Flags:
 ;  -DUSE_CONFIG_OVERRIDE
 ; lwIP 1.4 (Default)
@@ -100,15 +104,20 @@ monitor_speed             = 115200
 ;  -DVTABLES_IN_DRAM
 ; VTABLES in IRAM
 ;  -DVTABLES_IN_IRAM
-
+; NO_EXTRA_4K_HEAP - this forces the default NONOS-SDK user's heap location
+;     Default currently overlaps cont stack (Arduino) with sys stack (System)
+;     to save up-to 4 kB of heap. (starting core_2.4.2)
+; ESP8266_DISABLE_EXTRA4K  - Calls disable_extra4k_at_link_time() from setup
+;                            to force the linker keep user's stack in user ram.
+; CONT_STACKSIZE to set the 'cont' (Arduino) stack size. Default = 4096
 
 [common]
 board_build.f_cpu         = 80000000L
-build_unflags             = ""
+build_unflags             =
 build_flags               = -D BUILD_GIT='"${sysenv.TRAVIS_TAG}"'
                    -D NDEBUG
                    -lstdc++ -lsupc++
-                   -DPIO_FRAMEWORK_ARDUINO_LWIP2_LOW_MEMORY
+                   -DPIO_FRAMEWORK_ARDUINO_LWIP_HIGHER_BANDWIDTH
                    -DVTABLES_IN_FLASH
 
 
@@ -119,18 +128,20 @@ lib_archive               = false
 upload_speed              = 460800
 framework                 = arduino
 board                     = esp12e
-platform                  = ${core_2_4_2.platform}
+platform                  = ${core_2_4_1.platform}
 monitor_speed             = 115200
 
-
 [normal]
 platform                  = ${common.platform}
+build_flags               =
 
 [testing]
-platform                  = ${core_2_4_2.platform}
+platform                  = ${core_2_4_1.platform}
+build_flags               = -DPLUGIN_BUILD_TESTING
 
 [dev]
-platform                  = ${core_2_4_2.platform}
+platform                  = ${core_2_4_1.platform}
+build_flags               = -DPLUGIN_BUILD_DEV
 
 [ir]
 lib_ignore                = ESP32_ping, ESP32WebServer
@@ -158,7 +169,7 @@ board_build.flash_mode    = ${esp8266_1M.board_build.flash_mode}
 board_build.f_cpu         = ${esp8266_1M.board_build.f_cpu}
 build_unflags             = ${common.build_unflags}
 build_flags               = ${common.build_flags} -Wl,-Tesp8266.flash.1m128.ld
-platform                  = ${core_2_4_2.platform}
+platform                  = ${core_2_4_1.platform}
 
 [Sonoff_8285]
 board_upload.maximum_size = ${esp8285_1M.board_upload.maximum_size}
@@ -167,7 +178,7 @@ board_build.f_cpu         = ${esp8285_1M.board_build.f_cpu}
 board                     = ${esp8285_1M.board}
 build_unflags             = ${esp8285_1M.build_unflags}
 build_flags               = ${esp8285_1M.build_flags}
-platform                  = ${core_2_4_2.platform}
+platform                  = ${core_2_4_1.platform}
 
 [espWroom2M]
 board_build.flash_mode    = dout
@@ -193,6 +204,25 @@ build_flags               = ${common.build_flags} -Wl,-Tesp8266.flash.4m1m.ld
 ; Will probably not work, not tested and guaranteed to take a few hours time of some     ;
 ; still trying to build the version without reading this warning.                        ;
 ; *********************************************************************
+[env:esp-wrover-kit_test_1M8_partition]
+platform                  = ${core_esp32.platform}
+board                     = esp-wrover-kit
+build_unflags             = ${core_esp32.build_unflags}
+build_flags               = ${core_esp32.build_flags}  -DPLUGIN_SET_TEST_ESP32
+lib_deps                  = ${core_esp32.lib_deps}
+lib_ignore                = ${core_esp32.lib_ignore}
+lib_ldf_mode              = ${common.lib_ldf_mode}
+lib_archive               = ${common.lib_archive}
+framework                 = ${common.framework}
+upload_speed              = ${common.upload_speed}
+upload_protocol           = ftdi
+debug_tool                = ftdi
+monitor_speed             = ${common.monitor_speed}
+board_build.partitions    = esp32_partition_app1810k_spiffs316k.csv
+board_upload.maximum_size = 1900544
+debug_extra_cmds          = break Misc.ino:3011
+
+
 [env:esp32dev]
 platform                  = ${core_esp32.platform}
 board                     = esp32dev
@@ -206,6 +236,21 @@ framework                 = ${common.framework}
 upload_speed              = ${common.upload_speed}
 monitor_speed             = ${common.monitor_speed}
 
+[env:esp32test_1M8_partition]
+platform                  = ${core_esp32.platform}
+board                     = esp32dev
+build_unflags             = ${core_esp32.build_unflags}
+build_flags               = ${core_esp32.build_flags}  -DPLUGIN_SET_TEST_ESP32
+lib_deps                  = ${core_esp32.lib_deps}
+lib_ignore                = ${core_esp32.lib_ignore}
+lib_ldf_mode              = ${common.lib_ldf_mode}
+lib_archive               = ${common.lib_archive}
+framework                 = ${common.framework}
+upload_speed              = ${common.upload_speed}
+monitor_speed             = ${common.monitor_speed}
+board_build.partitions    = esp32_partition_app1810k_spiffs316k.csv
+board_upload.maximum_size = 1900544
+
 
 ;;; NORMAL (STABLE) ; ****; ****; ****; ****; ****; ****; ****; ****; ****; ****; ****;;;;
 ; normal version with stable plugins                                                     ;
@@ -226,7 +271,7 @@ board_upload.maximum_size = ${esp8266_1M.board_upload.maximum_size}
 board_build.f_cpu         = ${esp8266_1M.board_build.f_cpu}
 board_build.flash_mode    = ${esp8266_1M.board_build.flash_mode}
 build_unflags             = ${esp8266_1M.build_unflags}
-build_flags               = ${esp8266_1M.build_flags}
+build_flags               = ${esp8266_1M.build_flags} ${normal.build_flags}
 
 ; NORMAL: 1024k for esp8285 ----------------------
 [env:normal_ESP8285_1024]
@@ -243,7 +288,7 @@ board_upload.maximum_size = ${esp8285_1M.board_upload.maximum_size}
 board_build.f_cpu         = ${esp8285_1M.board_build.f_cpu}
 board_build.flash_mode    = ${esp8285_1M.board_build.flash_mode}
 build_unflags             = ${esp8285_1M.build_unflags}
-build_flags               = ${esp8285_1M.build_flags}
+build_flags               = ${esp8285_1M.build_flags} ${normal.build_flags}
 
 ; NORMAL: 2048k version --------------------------
 [env:normal_WROOM02_2048]
@@ -259,7 +304,7 @@ board                     = ${espWroom2M.board}
 board_build.f_cpu         = ${espWroom2M.board_build.f_cpu}
 board_build.flash_mode    = ${espWroom2M.board_build.flash_mode}
 build_unflags             = ${espWroom2M.build_unflags}
-build_flags               = ${espWroom2M.build_flags}
+build_flags               = ${espWroom2M.build_flags} ${normal.build_flags}
 
 ; NORMAL: 4096k version --------------------------
 [env:normal_ESP8266_4096]
@@ -275,7 +320,7 @@ board                     = ${common.board}
 board_build.f_cpu         = ${esp8266_4M.board_build.f_cpu}
 board_build.flash_mode    = ${esp8266_4M.board_build.flash_mode}
 build_unflags             = ${esp8266_4M.build_unflags}
-build_flags               = ${esp8266_4M.build_flags}
+build_flags               = ${esp8266_4M.build_flags} ${normal.build_flags}
 
 
 ; NORMAL IR: 4096k version --------------------------
@@ -293,7 +338,7 @@ board                     = ${common.board}
 board_build.f_cpu         = ${esp8266_4M.board_build.f_cpu}
 board_build.flash_mode    = ${esp8266_4M.board_build.flash_mode}
 build_unflags             = ${esp8266_4M.build_unflags}
-build_flags               = ${esp8266_4M.build_flags} -DPLUGIN_BUILD_NORMAL_IR
+build_flags               = ${esp8266_4M.build_flags} ${normal.build_flags} -DPLUGIN_BUILD_NORMAL_IR
 
 
 ;;; TESTING ; ****; ****; ****; ****; ****; ****; ****; ****; ****; ****; ****; ****; ****
@@ -315,7 +360,7 @@ board_upload.maximum_size = ${esp8266_1M.board_upload.maximum_size}
 board_build.flash_mode    = ${esp8266_1M.board_build.flash_mode}
 board_build.f_cpu         = ${esp8266_1M.board_build.f_cpu}
 build_unflags             = ${esp8266_1M.build_unflags}
-build_flags               = ${esp8266_1M.build_flags} -D PLUGIN_BUILD_TESTING
+build_flags               = ${esp8266_1M.build_flags} ${testing.build_flags}
 
 ; TEST: 1024k for esp8285 ------------------------
 [env:test_ESP8285_1024]
@@ -332,7 +377,7 @@ board_build.f_cpu         = ${esp8285_1M.board_build.f_cpu}
 board_build.flash_mode    = ${esp8285_1M.board_build.flash_mode}
 board                     = ${esp8285_1M.board}
 build_unflags             = ${esp8285_1M.build_unflags}
-build_flags               = ${esp8285_1M.build_flags} -D PLUGIN_BUILD_TESTING
+build_flags               = ${esp8285_1M.build_flags} ${testing.build_flags}
 
 ; TEST: 2048k version ----------------------------
 [env:test_WROOM02_2048]
@@ -348,7 +393,7 @@ board_build.f_cpu         = ${espWroom2M.board_build.f_cpu}
 board_build.flash_mode    = ${espWroom2M.board_build.flash_mode}
 board                     = ${espWroom2M.board}
 build_unflags             = ${espWroom2M.build_unflags}
-build_flags               = ${espWroom2M.build_flags} -D PLUGIN_BUILD_TESTING
+build_flags               = ${espWroom2M.build_flags} ${testing.build_flags}
 
 ; TEST: 4096k version ----------------------------
 [env:test_ESP8266_4096]
@@ -364,7 +409,7 @@ monitor_speed             = ${common.monitor_speed}
 board_build.f_cpu         = ${esp8266_4M.board_build.f_cpu}
 board_build.flash_mode    = ${esp8266_4M.board_build.flash_mode}
 build_unflags             = ${esp8266_4M.build_unflags}
-build_flags               = ${esp8266_4M.build_flags} -D PLUGIN_BUILD_TESTING
+build_flags               = ${esp8266_4M.build_flags} ${testing.build_flags}
 
 ; TEST: 4096k version + FEATURE_ADC_VCC ----------
 [env:test_ESP8266_4096_VCC]
@@ -380,7 +425,7 @@ monitor_speed             = ${common.monitor_speed}
 board_build.f_cpu         = ${esp8266_4M.board_build.f_cpu}
 board_build.flash_mode    = ${esp8266_4M.board_build.flash_mode}
 build_unflags             = ${esp8266_4M.build_unflags}
-build_flags               = ${esp8266_4M.build_flags} -D PLUGIN_BUILD_TESTING -D FEATURE_ADC_VCC=true
+build_flags               = ${esp8266_4M.build_flags} ${testing.build_flags} -D FEATURE_ADC_VCC=true
 
 
 
@@ -403,7 +448,7 @@ board_build.f_cpu         = ${esp8266_1M.board_build.f_cpu}
 board_build.flash_mode    = ${esp8266_1M.board_build.flash_mode}
 board                     = ${common.board}
 build_unflags             = ${esp8266_1M.build_unflags}
-build_flags               = ${esp8266_1M.build_flags} -D PLUGIN_BUILD_DEV
+build_flags               = ${esp8266_1M.build_flags} ${dev.build_flags}
 
 ; DEV: 1024k for esp8285 -------------------------
 [env:dev_ESP8285_1024]
@@ -420,7 +465,7 @@ board_build.f_cpu         = ${esp8285_1M.board_build.f_cpu}
 board_build.flash_mode    = ${esp8285_1M.board_build.flash_mode}
 board                     = ${esp8285_1M.board}
 build_unflags             = ${esp8285_1M.build_unflags}
-build_flags               = ${esp8285_1M.build_flags} -D PLUGIN_BUILD_DEV
+build_flags               = ${esp8285_1M.build_flags} ${dev.build_flags}
 
 ; DEV: 2048k version -----------------------------
 [env:dev_WROOM02_2048]
@@ -436,7 +481,7 @@ board_build.f_cpu         = ${espWroom2M.board_build.f_cpu}
 board_build.flash_mode    = ${espWroom2M.board_build.flash_mode}
 board                     = ${espWroom2M.board}
 build_unflags             = ${espWroom2M.build_unflags}
-build_flags               = ${espWroom2M.build_flags} -D PLUGIN_BUILD_DEV
+build_flags               = ${espWroom2M.build_flags} ${dev.build_flags}
 
 ; DEV : 4096k version ----------------------------
 [env:dev_ESP8266_4096]
@@ -452,7 +497,7 @@ monitor_speed             = ${common.monitor_speed}
 board_build.f_cpu         = ${esp8266_4M.board_build.f_cpu}
 board_build.flash_mode    = ${esp8266_4M.board_build.flash_mode}
 build_unflags             = ${esp8266_4M.build_unflags}
-build_flags               = ${esp8266_4M.build_flags} -D PLUGIN_BUILD_DEV
+build_flags               = ${esp8266_4M.build_flags} ${dev.build_flags}
 
 
 
@@ -475,7 +520,7 @@ board_upload.maximum_size = ${esp8266_1M.board_upload.maximum_size}
 board_build.f_cpu         = ${esp8266_1M.board_build.f_cpu}
 board_build.flash_mode    = ${esp8266_1M.board_build.flash_mode}
 build_unflags             = ${esp8266_1M.build_unflags}
-build_flags               = ${esp8266_1M.build_flags} -D PLUGIN_BUILD_DEV -D FLASH_QUIRK_WRITE_0_TO_1
+build_flags               = ${esp8266_1M.build_flags} ${dev.build_flags} -D FLASH_QUIRK_WRITE_0_TO_1
 
 ; DEV+PUYA : 1024k version + FEATURE_ADC_VCC -----
 [env:dev_ESP8266PUYA_1024_VCC]
@@ -492,7 +537,7 @@ board_upload.maximum_size = ${esp8266_1M.board_upload.maximum_size}
 board_build.f_cpu         = ${esp8266_1M.board_build.f_cpu}
 board_build.flash_mode    = ${esp8266_1M.board_build.flash_mode}
 build_unflags             = ${esp8266_1M.build_unflags}
-build_flags               = ${esp8266_1M.build_flags} -D PLUGIN_BUILD_DEV -D FLASH_QUIRK_WRITE_0_TO_1 -D FEATURE_ADC_VCC=true
+build_flags               = ${esp8266_1M.build_flags} ${dev.build_flags} -D FLASH_QUIRK_WRITE_0_TO_1 -D FEATURE_ADC_VCC=true
 
 
 

preflight.sh

@@ -0,0 +1,36 @@
+#!/bin/bash
+
+# Check for documentation only changes.
+# Source: https://github.com/dev-id/Magic-Spoiler/blob/8111a06ab6682e020169991d5e2aa4fa503d787f/preflight.sh
+
+set -e
+MAIN_BRANCH="mega"
+MD=".md$"
+DOCS="^docs/"
+
+CHANGED_FILES=`git diff --name-only ${MAIN_BRANCH}...${TRAVIS_COMMIT}`
+ONLY_READMES=True
+DOCUMENTATION_CHANGED=False
+
+for CHANGED_FILE in $CHANGED_FILES; do
+  if ! [[ $CHANGED_FILE =~ $MD || $CHANGED_FILE =~ $DOCS ]]; then
+    ONLY_READMES=False
+#    break
+  else
+    DOCUMENTATION_CHANGED=True
+  fi
+done
+
+#if [[ $ONLY_READMES == True ]]; then
+#  echo "Only documentation files found, exiting."
+#  travis_terminate 0
+#  exit 1
+#else
+#  echo "Non-documentation files found, continuing with build."
+#fi
+
+if [[ $DOCUMENTATION_CHANGED == True ]]; then
+  echo "Generate documentation."
+  cd docs
+  make html
+fi

src/Command.ino

@@ -131,7 +131,8 @@ String doExecuteCommand(const char * cmd, struct EventStruct *event, const char*
 	  COMMAND_CASE("taskclear"              , Command_Task_Clear);                 // Tasks.h
 	  COMMAND_CASE("taskclearall"           , Command_Task_ClearAll);              // Tasks.h
 	  COMMAND_CASE("taskrun"                , Command_Task_Run);                   // Tasks.h
-	  COMMAND_CASE("taskvalueset"           , Command_Task_ValueSet);              // Tasks.h
+    COMMAND_CASE("taskvalueset"           , Command_Task_ValueSet);              // Tasks.h
+    COMMAND_CASE("taskvaluetoggle"        , Command_Task_ValueToggle);           // Tasks.h
 	  COMMAND_CASE("taskvaluesetandrun"     , Command_Task_ValueSetAndRun);        // Tasks.h
 	  COMMAND_CASE("timerpause"             , Command_Timer_Pause);                // Timers.h
 	  COMMAND_CASE("timerresume"            , Command_Timer_Resume);               // Timers.h
@@ -216,11 +217,11 @@ void ExecuteCommand(byte source, const char *Line)
 	// since commands can originate from anywhere.
 	TempEvent.Source = source;
 	GetArgv(Line, cmd, 1);
-	if (GetArgv(Line, TmpStr1, 2)) TempEvent.Par1 = str2int(TmpStr1);
-	if (GetArgv(Line, TmpStr1, 3)) TempEvent.Par2 = str2int(TmpStr1);
-	if (GetArgv(Line, TmpStr1, 4)) TempEvent.Par3 = str2int(TmpStr1);
-	if (GetArgv(Line, TmpStr1, 5)) TempEvent.Par4 = str2int(TmpStr1);
-	if (GetArgv(Line, TmpStr1, 6)) TempEvent.Par5 = str2int(TmpStr1);
+  if (GetArgv(Line, TmpStr1, 2)) TempEvent.Par1 = CalculateParam(TmpStr1);
+	if (GetArgv(Line, TmpStr1, 3)) TempEvent.Par2 = CalculateParam(TmpStr1);
+	if (GetArgv(Line, TmpStr1, 4)) TempEvent.Par3 = CalculateParam(TmpStr1);
+	if (GetArgv(Line, TmpStr1, 5)) TempEvent.Par4 = CalculateParam(TmpStr1);
+	if (GetArgv(Line, TmpStr1, 6)) TempEvent.Par5 = CalculateParam(TmpStr1);
 
   if (source == VALUE_SOURCE_WEB_FRONTEND) {
     // Must run immediately, to see result in web frontend

src/Commands/Common.h

@@ -18,36 +18,6 @@ bool IsNumeric(char * source)
 	return result;
 }
 
-bool safeReadStringUntil(Stream &input, String &str, char terminator, unsigned int maxSize = 1024, unsigned int timeout = 1000)
-{
-	int c;
-	const unsigned long timer = millis() + timeout;
-	str = "";
-
-	do {
-		//read character
-		c = input.read();
-		if (c >= 0) {
-			//found terminator, we're ok
-			if (c == terminator) {
-				return(true);
-			}
-			//found character, add to string
-			else{
-				str += char(c);
-				//string at max size?
-				if (str.length() >= maxSize) {
-					addLog(LOG_LEVEL_ERROR, F("Not enough bufferspace to read all input data!"));
-					return(false);
-				}
-			}
-		}
-		yield();
-	} while (!timeOutReached(timer));
-
-	addLog(LOG_LEVEL_ERROR, F("Timeout while reading input data!"));
-	return(false);
-}
 
 String Command_GetORSetIP(struct EventStruct *event,
 	      const __FlashStringHelper *targetDescription,

src/Commands/HTTP.h

@@ -23,32 +23,17 @@ String Command_HTTP_SendToHTTP(struct EventStruct *event, const char* Line)
 			addLog(LOG_LEVEL_DEBUG, log);
 		}
 		WiFiClient client;
-		if (client.connect(host.c_str(), port.toInt())) {
-			String reply = F("GET ");
-			reply += path;
-			reply += F(" HTTP/1.1\r\n");
-			reply += F("Host: ");
-			reply += host;
-			reply += F("\r\n");
-			reply += F("Connection: close\r\n\r\n");
-			addLog(LOG_LEVEL_DEBUG, reply);
-			client.print(reply);
-
-			unsigned long timer = millis() + 200;
-			while (!client.available() && !timeOutReached(timer))
-				delay(1);
-
-			while (client.available()) {
-				// String line = client.readStringUntil('\n');
-				String line;
-				safeReadStringUntil(client, line, '\n');
-
-				if (line.substring(0, 15) == F("HTTP/1.1 200 OK"))
-					addLog(LOG_LEVEL_DEBUG, line);
-				delay(1);
+		const int port_int = port.toInt();
+		const bool connected = client.connect(host.c_str(), port_int) == 1;
+		if (connected) {
+			String hostportString = host;
+			if (port_int != 0 && port_int != 80) {
+				hostportString += ':';
+				hostportString += port_int;
 			}
-			client.flush();
-			client.stop();
+			String request = do_create_http_request(hostportString, F("GET"), path);
+			addLog(LOG_LEVEL_DEBUG, request);
+			send_via_http(F("Command_HTTP_SendToHTTP"), client, request, false);
 		}
 	}
 	return return_command_success();

src/Commands/Tasks.h

@@ -30,6 +30,15 @@ String Command_Task_ValueSet(struct EventStruct *event, const char* Line)
 	return return_command_success();
 }
 
+String Command_Task_ValueToggle(struct EventStruct *event, const char* Line)
+{
+	const int result = round(UserVar[(VARS_PER_TASK * (event->Par1 - 1)) + event->Par2 - 1]);
+	if (result == 0 || result == 1) {
+	  UserVar[(VARS_PER_TASK * (event->Par1 - 1)) + event->Par2 - 1] = !result;
+	}
+	return return_command_success();
+}
+
 String Command_Task_ValueSetAndRun(struct EventStruct *event, const char* Line)
 {
 	char TmpStr1[INPUT_COMMAND_SIZE];

src/Controller.ino

@@ -15,6 +15,8 @@ void sendData(struct EventStruct *event)
 //  if (!Settings.TaskDeviceSendData[event->TaskIndex])
 //    return false;
 
+/*
+  // Disabed for now, using buffers at controller side.
   if (Settings.MessageDelay != 0)
   {
     const long dif = timePassedSince(lastSend);
@@ -31,6 +33,7 @@ void sendData(struct EventStruct *event)
       //   backgroundtasks();
     }
   }
+*/
 
   LoadTaskSettings(event->TaskIndex); // could have changed during background tasks.
 
@@ -39,21 +42,25 @@ void sendData(struct EventStruct *event)
     event->ControllerIndex = x;
     event->idx = Settings.TaskDeviceID[x][event->TaskIndex];
     if (Settings.TaskDeviceSendData[event->ControllerIndex][event->TaskIndex] &&
-        Settings.ControllerEnabled[event->ControllerIndex] && Settings.Protocol[event->ControllerIndex])
+        Settings.ControllerEnabled[event->ControllerIndex] &&
+        Settings.Protocol[event->ControllerIndex])
     {
       event->ProtocolIndex = getProtocolIndex(Settings.Protocol[event->ControllerIndex]);
       if (validUserVar(event)) {
         CPlugin_ptr[event->ProtocolIndex](CPLUGIN_PROTOCOL_SEND, event, dummyString);
       } else {
-        String log = F("Invalid value detected for controller ");
-        String controllerName;
-        CPlugin_ptr[event->ProtocolIndex](CPLUGIN_GET_DEVICENAME, event, controllerName);
-        log += controllerName;
-        addLog(LOG_LEVEL_DEBUG, log);
+        if (loglevelActiveFor(LOG_LEVEL_DEBUG)) {
+          String log = F("Invalid value detected for controller ");
+          String controllerName;
+          CPlugin_ptr[event->ProtocolIndex](CPLUGIN_GET_DEVICENAME, event, controllerName);
+          log += controllerName;
+          addLog(LOG_LEVEL_DEBUG, log);
+        }
       }
     }
   }
 
+  // FIXME TD-er: This PLUGIN_EVENT_OUT seems to be unused.
   PluginCall(PLUGIN_EVENT_OUT, event, dummyString);
   lastSend = millis();
   STOP_TIMER(SEND_DATA_STATS);
@@ -126,7 +133,7 @@ bool MQTTConnect(int controller_idx)
 {
   ++mqtt_reconnect_count;
   ControllerSettingsStruct ControllerSettings;
-  LoadControllerSettings(controller_idx, (byte*)&ControllerSettings, sizeof(ControllerSettings));
+  LoadControllerSettings(controller_idx, ControllerSettings);
   if (!ControllerSettings.checkHostReachable(true))
     return false;
   if (MQTTclient.connected()) {
@@ -134,6 +141,7 @@ bool MQTTConnect(int controller_idx)
     updateMQTTclient_connected();
   }
   mqtt = WiFiClient(); // workaround see: https://github.com/esp8266/Arduino/issues/4497#issuecomment-373023864
+  mqtt.setTimeout(ControllerSettings.ClientTimeout);
   if (ControllerSettings.UseDNS) {
     MQTTclient.setServer(ControllerSettings.getHost().c_str(), ControllerSettings.Port);
   } else {
@@ -154,6 +162,15 @@ bool MQTTConnect(int controller_idx)
     clientid = F("ESPClient_");
     clientid += WiFi.macAddress();
   }
+  // Work-around for 'lost connections' to the MQTT broker.
+  // If the broker thinks the connection is still alive, a reconnect from the
+  // client will be refused.
+  // To overcome this issue, append the number of reconnects to the client ID to
+  // make it different from the previous one.
+  if (wifi_reconnects >= 1) {
+    clientid += F("_");
+    clientid += wifi_reconnects;
+  }
 
   String LWTTopic = ControllerSettings.MQTTLwtTopic;
   if(LWTTopic.length() == 0)
@@ -265,12 +282,33 @@ void SendStatus(byte source, String status)
 
 boolean MQTTpublish(int controller_idx, const char* topic, const char* payload, boolean retained)
 {
-  if (MQTTclient.publish(topic, payload, retained)) {
+  const bool success = MQTTDelayHandler.addToQueue(MQTT_queue_element(controller_idx, topic, payload, retained));
+  scheduleNextMQTTdelayQueue();
+  return success;
+}
+
+void scheduleNextMQTTdelayQueue() {
+  scheduleNextDelayQueue(TIMER_MQTT_DELAY_QUEUE, MQTTDelayHandler.getNextScheduleTime());
+}
+
+void processMQTTdelayQueue() {
+  START_TIMER;
+  MQTT_queue_element element;
+  if (!MQTTDelayHandler.getNext(element)) return;
+  if (MQTTclient.publish(element._topic.c_str(), element._payload.c_str(), element._retained)) {
     setIntervalTimerOverride(TIMER_MQTT, 10); // Make sure the MQTT is being processed as soon as possible.
-    return true;
+    MQTTDelayHandler.markProcessed(true);
+  } else {
+    MQTTDelayHandler.markProcessed(false);
+    if (loglevelActiveFor(LOG_LEVEL_DEBUG)) {
+      String log = F("MQTT : process MQTT queue not published, ");
+      log += MQTTDelayHandler.sendQueue.size();
+      log += F(" items left in queue");
+      addLog(LOG_LEVEL_DEBUG, log);
+    }
   }
-  addLog(LOG_LEVEL_DEBUG, F("MQTT : publish failed"));
-  return false;
+  scheduleNextMQTTdelayQueue();
+  STOP_TIMER(MQTT_DELAY_QUEUE);
 }
 
 /*********************************************************************************************\
@@ -281,7 +319,7 @@ void MQTTStatus(String& status)
   ControllerSettingsStruct ControllerSettings;
   int enabledMqttController = firstEnabledMQTTController();
   if (enabledMqttController >= 0) {
-    LoadControllerSettings(enabledMqttController, (byte*)&ControllerSettings, sizeof(ControllerSettings));
+    LoadControllerSettings(enabledMqttController, ControllerSettings);
     String pubname = ControllerSettings.Subscribe;
     pubname.replace(F("/#"), F("/status"));
     parseSystemVariables(pubname, false);

src/ESPEasy-Globals.h

@@ -11,7 +11,6 @@
 // Set default configuration settings if you want (not mandatory)
 // You can always change these during runtime and save to eeprom
 // After loading firmware, issue a 'reset' command to load the defaults.
-
 // --- Basic Config Settings ------------------------------------------------------------------------
 #define DEFAULT_NAME        "ESP_Easy"                  // Enter your device friendly name
 #define UNIT                            0                                       // Unit Number
@@ -69,6 +68,9 @@
 #if defined(ESP32)
   #define USE_RTOS_MULTITASKING
 #endif
+#ifdef M5STACK_ESP
+//  #include <M5Stack.h>
+#endif
 
 #define DEFAULT_USE_RULES                       false   // (true|false) Enable Rules?
 
@@ -77,7 +79,7 @@
 #define DEFAULT_MQTT_LWT_TOPIC                  ""      // Default lwt topic
 #define DEFAULT_MQTT_LWT_CONNECT_MESSAGE        "Connected" // Default lwt message
 #define DEFAULT_MQTT_LWT_DISCONNECT_MESSAGE     "Connection Lost" // Default lwt message
-#define DEFAULT_MQTT_USE_UNITNANE_AS_CLIENTID   0
+#define DEFAULT_MQTT_USE_UNITNAME_AS_CLIENTID   0
 
 #define DEFAULT_USE_NTP                         false   // (true|false) Use NTP Server
 #define DEFAULT_NTP_HOST                        ""              // NTP Server Hostname
@@ -189,6 +191,8 @@
 
 #define MAX_FLASHWRITES_PER_DAY           100 // per 24 hour window
 #define INPUT_COMMAND_SIZE                240 // Affects maximum command length in rules and other commands
+// FIXME TD-er: INPUT_COMMAND_SIZE is also used in commands where simply a check for valid parameter is needed
+// and some may need less memory. (which is stack allocated)
 
 #define NODE_TYPE_ID_ESP_EASY_STD           1
 #define NODE_TYPE_ID_ESP_EASYM_STD         17
@@ -202,6 +206,32 @@
 #define TIMER_30SEC                         4
 #define TIMER_MQTT                          5
 #define TIMER_STATISTICS                    6
+#define TIMER_MQTT_DELAY_QUEUE              7
+#define TIMER_C001_DELAY_QUEUE              8
+#define TIMER_C003_DELAY_QUEUE              9
+#define TIMER_C004_DELAY_QUEUE             10
+#define TIMER_C007_DELAY_QUEUE             11
+#define TIMER_C008_DELAY_QUEUE             12
+#define TIMER_C009_DELAY_QUEUE             13
+#define TIMER_C010_DELAY_QUEUE             14
+#define TIMER_C011_DELAY_QUEUE             15
+#define TIMER_C012_DELAY_QUEUE             16
+#define TIMER_C013_DELAY_QUEUE             17
+
+// Minimum delay between messages for a controller to send in msec.
+#define CONTROLLER_DELAY_QUEUE_DELAY_MAX   3600000
+#define CONTROLLER_DELAY_QUEUE_DELAY_DFLT  100
+// Queue length for controller messages not yet sent.
+#define CONTROLLER_DELAY_QUEUE_DEPTH_MAX   25
+#define CONTROLLER_DELAY_QUEUE_DEPTH_DFLT  10
+// Number of retries to send a message by a controller.
+// N.B. Retries without a connection to wifi do not count as retry.
+#define CONTROLLER_DELAY_QUEUE_RETRY_MAX   10
+#define CONTROLLER_DELAY_QUEUE_RETRY_DFLT  10
+// Timeout of the client in msec.
+#define CONTROLLER_CLIENTTIMEOUT_MAX 1000
+#define CONTROLLER_CLIENTTIMEOUT_DFLT 300
+
 
 #define PLUGIN_INIT_ALL                     1
 #define PLUGIN_INIT                         2
@@ -262,6 +292,7 @@
 #define NPLUGIN_NOT_FOUND                 255
 
 
+#define LOG_LEVEL_NONE                      0
 #define LOG_LEVEL_ERROR                     1
 #define LOG_LEVEL_INFO                      2
 #define LOG_LEVEL_DEBUG                     3
@@ -275,7 +306,11 @@
 #if defined(PLUGIN_BUILD_TESTING) || defined(PLUGIN_BUILD_DEV)
   #define DEVICES_MAX                      75
 #else
-  #define DEVICES_MAX                      50
+  #ifdef ESP32
+    #define DEVICES_MAX                      75
+  #else
+    #define DEVICES_MAX                      50
+  #endif
 #endif
 
 #if defined(ESP8266)
@@ -295,7 +330,7 @@
 #define PLUGIN_EXTRACONFIGVAR_MAX          16
 #define CPLUGIN_MAX                        16
 #define NPLUGIN_MAX                         4
-#define UNIT_MAX                           32 // Only relevant for UDP unicast message 'sweeps' and the nodelist.
+#define UNIT_MAX                          254 // unit 255 = broadcast
 #define RULES_TIMER_MAX                     8
 #define PINSTATE_TABLE_MAX                 32
 #define RULES_MAX_SIZE                   2048
@@ -303,6 +338,8 @@
 #define RULESETS_MAX                        4
 #define RULES_BUFFER_SIZE                  64
 #define NAME_FORMULA_LENGTH_MAX            40
+#define RULES_IF_MAX_NESTING_LEVEL          4
+
 #define UDP_PACKETSIZE_MAX               2048
 
 #define PIN_MODE_UNDEFINED                  0
@@ -335,6 +372,9 @@
 #define SENSOR_TYPE_LONG                   20
 #define SENSOR_TYPE_WIND                   21
 
+#define UNIT_NUMBER_MAX                  9999  // Stored in Settings.Unit
+#define DOMOTICZ_MAX_IDX            999999999  // Looks like it is an unsigned int, so could be up to 4 bln.
+
 #define VALUE_SOURCE_SYSTEM                 1
 #define VALUE_SOURCE_SERIAL                 2
 #define VALUE_SOURCE_HTTP                   3
@@ -370,6 +410,23 @@
   #define CONFIG_FILE_SIZE               131072
 #endif
 
+// Forward declaration
+struct ControllerSettingsStruct;
+void scheduleNextDelayQueue(unsigned long id, unsigned long nextTime);
+String LoadControllerSettings(int ControllerIndex, ControllerSettingsStruct& controller_settings);
+String get_formatted_Controller_number(int controller_index);
+bool loglevelActiveFor(byte logLevel);
+void addToLog(byte loglevel, const String& string);
+void addToLog(byte logLevel, const __FlashStringHelper* flashString);
+void statusLED(boolean traffic);
+void backgroundtasks();
+uint32_t getCurrentFreeStack();
+uint32_t getFreeStackWatermark();
+bool canYield();
+
+bool getBitFromUL(uint32_t number, byte bitnr);
+void setBitToUL(uint32_t& number, byte bitnr, bool value);
+
 enum SettingsType {
   BasicSettings_Type = 0,
   TaskSettings_Type,
@@ -412,6 +469,7 @@ bool showSettingsFileLayout = false;
 #include "I2CTypes.h"
 #include <I2Cdev.h>
 #include <map>
+#include <deque>
 
 #define FS_NO_GLOBALS
 #if defined(ESP8266)
@@ -479,10 +537,10 @@ bool showSettingsFileLayout = false;
   #define FILE_RULES        "/rules1.txt"
   #include <WiFi.h>
   #include  "esp32_ping.h"
-  #include <ESP32WebServer.h>
+  #include <WebServer.h>
   #include "SPIFFS.h"
   #include <rom/rtc.h>
-  ESP32WebServer WebServer(80);
+  WebServer WebServer(80);
   #ifdef FEATURE_MDNS
     #include <ESPmDNS.h>
   #endif
@@ -550,6 +608,9 @@ class TimingStats;
 #define LOADFILE_STATS  0
 #define LOOP_STATS      1
 
+/*********************************************************************************************\
+ * CRCStruct
+\*********************************************************************************************/
 struct CRCStruct{
   char compileTimeMD5[16+32+1]= "MD5_MD5_MD5_MD5_BoundariesOfTheSegmentsGoHere...";
   char binaryFilename[32+32+1]= "ThisIsTheDummyPlaceHolderForTheBinaryFilename64ByteLongFilenames";
@@ -580,7 +641,9 @@ int16_t I2C_readS16_reg(uint8_t i2caddr, byte reg);
 int16_t I2C_readS16_LE_reg(uint8_t i2caddr, byte reg);
 I2Cdev i2cdev;
 
-
+/*********************************************************************************************\
+ * SecurityStruct
+\*********************************************************************************************/
 struct SecurityStruct
 {
   SecurityStruct() {
@@ -612,23 +675,56 @@ struct SecurityStruct
   uint8_t       md5[16];
 } SecuritySettings;
 
+
+/*********************************************************************************************\
+ * SettingsStruct
+\*********************************************************************************************/
 struct SettingsStruct
 {
   SettingsStruct() {
     clearAll();
   }
 
+  // VariousBits1 defaults to 0, keep in mind when adding bit lookups.
+  bool appendUnitToHostname() {  return !getBitFromUL(VariousBits1, 1); }
+  void appendUnitToHostname(bool value) { setBitToUL(VariousBits1, 1, !value); }
+
+  void validate() {
+    if (UDPPort > 65535) UDPPort = 0;
+
+    if (Latitude  < -90.0  || Latitude > 90.0) Latitude = 0.0;
+    if (Longitude < -180.0 || Longitude > 180.0) Longitude = 0.0;
+    if (VariousBits1 > (1 << 30)) VariousBits1 = 0;
+  }
+
+  bool networkSettingsEmpty() {
+    return (IP[0] == 0 && Gateway[0] == 0 && Subnet[0] == 0 && DNS[0] == 0);
+  }
+
+  void clearNetworkSettings() {
+    for (byte i = 0; i < 4; ++i) {
+      IP[i] = 0;
+      Gateway[i] = 0;
+      Subnet[i] = 0;
+      DNS[i] = 0;
+    }
+  }
+
   void clearAll() {
     PID = 0;
     Version = 0;
     Build = 0;
     IP_Octet = 0;
     Unit = 0;
+    Name[0] = 0;
+    NTPHost[0] = 0;
     Delay = 0;
     Pin_i2c_sda = -1;
     Pin_i2c_scl = -1;
     Pin_status_led = -1;
     Pin_sd_cs = -1;
+    for (byte i = 0; i < 17; ++i) { PinBootStates[i] = 0; }
+    for (byte i = 0; i < 4; ++i) {  Syslog_IP[i] = 0; }
     UDPPort = 0;
     SyslogLevel = 0;
     SerialLogLevel = 0;
@@ -653,13 +749,17 @@ struct SettingsStruct
     Pin_status_led_Inversed = false;
     deepSleepOnFail = false;
     UseValueLogger = false;
+    ArduinoOTAEnable = false;
     DST_Start = 0;
     DST_End = 0;
     UseRTOSMultitasking = false;
     Pin_Reset = -1;
     SyslogFacility = DEFAULT_SYSLOG_FACILITY;
-    StructSize = 0;
+    StructSize = sizeof(SettingsStruct);
     MQTTUseUnitNameAsClientId = 0;
+    Latitude = 0.0;
+    Longitude = 0.0;
+    VariousBits1 = 0;
 
     for (byte i = 0; i < CONTROLLER_MAX; ++i) {
       Protocol[i] = 0;
@@ -672,6 +772,7 @@ struct SettingsStruct
     for (byte task = 0; task < TASKS_MAX; ++task) {
       clearTask(task);
     }
+    clearNetworkSettings();
   }
 
   void clearTask(byte task) {
@@ -786,6 +887,7 @@ struct SettingsStruct
   //TODO: document config.dat somewhere here
   float         Latitude;
   float         Longitude;
+  uint32_t      VariousBits1;
 
   // FIXME @TD-er: As discussed in #1292, the CRC for the settings is now disabled.
   // make sure crc is the last value in the struct
@@ -793,10 +895,35 @@ struct SettingsStruct
 //  uint8_t       ProgmemMd5[16]; // crc of the binary that last saved the struct to file.
 //  uint8_t       md5[16];
 } Settings;
+/*
+SettingsStruct* SettingsStruct_ptr = new SettingsStruct;
+SettingsStruct& Settings = *SettingsStruct_ptr;
+*/
 
+
+/*********************************************************************************************\
+ *  Analyze SettingsStruct and report inconsistencies
+ *  Not a member function to be able to use the F-macro
+\*********************************************************************************************/
+bool SettingsCheck(String& error) {
+  error = "";
+  if (!Settings.networkSettingsEmpty()) {
+    if (Settings.IP[0] == 0 || Settings.Gateway[0] == 0 || Settings.Subnet[0] == 0 || Settings.DNS[0] == 0) {
+      error += F("Error: Either fill all IP settings fields or leave all empty");
+    }
+  }
+
+  return error.length() == 0;
+}
+
+/*********************************************************************************************\
+ * ControllerSettingsStruct definition
+\*********************************************************************************************/
 struct ControllerSettingsStruct
 {
-  ControllerSettingsStruct() : UseDNS(false), Port(0) {
+  ControllerSettingsStruct() : UseDNS(false), Port(0),
+      MinimalTimeBetweenMessages(100), MaxQueueDepth(10), MaxRetry(10),
+      DeleteOldest(false), ClientTimeout(100), MustCheckReply(false) {
     for (byte i = 0; i < 4; ++i) {
       IP[i] = 0;
     }
@@ -816,6 +943,23 @@ struct ControllerSettingsStruct
   char          MQTTLwtTopic[129];
   char          LWTMessageConnect[129];
   char          LWTMessageDisconnect[129];
+  unsigned int  MinimalTimeBetweenMessages;
+  unsigned int  MaxQueueDepth;
+  unsigned int  MaxRetry;
+  boolean       DeleteOldest; // Action to perform when buffer full, delete oldest, or ignore newest.
+  unsigned int  ClientTimeout;
+  boolean       MustCheckReply; // When set to false, a sent message is considered always successful.
+
+  void validate() {
+    if (Port > 65535) Port = 0;
+    if (MinimalTimeBetweenMessages < 1  ||  MinimalTimeBetweenMessages > CONTROLLER_DELAY_QUEUE_DELAY_MAX)
+      MinimalTimeBetweenMessages = CONTROLLER_DELAY_QUEUE_DELAY_DFLT;
+    if (MaxQueueDepth > CONTROLLER_DELAY_QUEUE_DEPTH_MAX) MaxQueueDepth = CONTROLLER_DELAY_QUEUE_DEPTH_DFLT;
+    if (MaxRetry > CONTROLLER_DELAY_QUEUE_RETRY_MAX) MaxRetry = CONTROLLER_DELAY_QUEUE_RETRY_MAX;
+    if (ClientTimeout < 10 || ClientTimeout > CONTROLLER_CLIENTTIMEOUT_MAX) {
+      ClientTimeout = CONTROLLER_CLIENTTIMEOUT_DFLT;
+    }
+  }
 
   IPAddress getIP() const {
     IPAddress host(IP[0], IP[1], IP[2], IP[3]);
@@ -838,6 +982,7 @@ struct ControllerSettingsStruct
     if (!WiFiConnected(10)) {
       return false; // Not connected, so no use in wasting time to connect to a host.
     }
+    delay(1); // Make sure the Watchdog will not trigger a reset.
     if (quick && ipSet()) return true;
     if (UseDNS) {
       if (!updateIPcache()) {
@@ -866,21 +1011,22 @@ struct ControllerSettingsStruct
     return false;
   }
 
+  // Returns 1 if successful, 0 if there was a problem resolving the hostname or port
   int beginPacket(WiFiUDP &client) {
     if (!checkHostReachable(true)) {
       return 0; // Host not reachable
     }
     byte retry = 2;
     int connected = 0;
-    while (retry > 0 && !connected) {
+    while (retry > 0 && connected == 0) {
       --retry;
       connected = client.beginPacket(getIP(), Port);
       if (connected != 0) return connected;
       if (!checkHostReachable(false))
-        return false;
+        return 0;
       delay(10);
     }
-    return false;
+    return 0;
   }
 
   String getHostPortString() const {
@@ -915,6 +1061,10 @@ private:
 
 };
 
+
+/*********************************************************************************************\
+ * NotificationSettingsStruct
+\*********************************************************************************************/
 struct NotificationSettingsStruct
 {
   NotificationSettingsStruct() : Port(0), Pin1(0), Pin2(0) {
@@ -942,6 +1092,10 @@ struct NotificationSettingsStruct
   //its safe to extend this struct, up to 4096 bytes, default values in config are 0
 };
 
+/*********************************************************************************************\
+ * ExtraTaskSettingsStruct
+\*********************************************************************************************/
+
 // This is only used by some plugins to store extra settings like formula descriptions.
 // These settings can only be active for one plugin, meaning they have to be loaded
 // over and over again from flash when another active plugin uses these values.
@@ -1015,6 +1169,9 @@ struct ExtraTaskSettingsStruct
   int16_t TaskDevicePluginConfig[PLUGIN_EXTRACONFIGVAR_MAX];
 } ExtraTaskSettings;
 
+/*********************************************************************************************\
+ * EventStruct
+\*********************************************************************************************/
 struct EventStruct
 {
   EventStruct() :
@@ -1058,21 +1215,26 @@ struct EventStruct
   byte *Data;
 };
 
+
+/*********************************************************************************************\
+ * LogStruct
+\*********************************************************************************************/
 #define LOG_STRUCT_MESSAGE_SIZE 128
 #ifdef ESP32
   #define LOG_STRUCT_MESSAGE_LINES 30
+  #define LOG_BUFFER_EXPIRE         30000  // Time after which a buffered log item is considered expired.
 #else
   #if defined(PLUGIN_BUILD_TESTING) || defined(PLUGIN_BUILD_DEV)
     #define LOG_STRUCT_MESSAGE_LINES 10
   #else
     #define LOG_STRUCT_MESSAGE_LINES 15
   #endif
+  #define LOG_BUFFER_EXPIRE         5000  // Time after which a buffered log item is considered expired.
 #endif
 
 struct LogStruct {
-    LogStruct() : write_idx(0), read_idx(0) {
+    LogStruct() : write_idx(0), read_idx(0), lastReadTimeStamp(0) {
       for (int i = 0; i < LOG_STRUCT_MESSAGE_LINES; ++i) {
-        Message[i].reserve(LOG_STRUCT_MESSAGE_SIZE);
         timeStamp[i] = 0;
         log_level[i] = 0;
       }
@@ -1090,6 +1252,7 @@ struct LogStruct {
       if (linelength > LOG_STRUCT_MESSAGE_SIZE-1)
         linelength = LOG_STRUCT_MESSAGE_SIZE-1;
       Message[write_idx] = "";
+      Message[write_idx].reserve(linelength);
       for (unsigned i = 0; i < linelength; ++i) {
         Message[write_idx] += *(line + i);
       }
@@ -1098,6 +1261,7 @@ struct LogStruct {
     // Read the next item and append it to the given string.
     // Returns whether new lines are available.
     bool get(String& output, const String& lineEnd) {
+      lastReadTimeStamp = millis();
       if (!isEmpty()) {
         read_idx = (read_idx + 1) % LOG_STRUCT_MESSAGE_LINES;
         output += formatLine(read_idx, lineEnd);
@@ -1106,6 +1270,7 @@ struct LogStruct {
     }
 
     String get_logjson_formatted(bool& logLinesAvailable, unsigned long& timestamp) {
+      lastReadTimeStamp = millis();
       logLinesAvailable = false;
       if (isEmpty()) {
         return "";
@@ -1119,31 +1284,15 @@ struct LogStruct {
       return output;
     }
 
-    bool get(String& output, const String& lineEnd, int line) {
-      int tmpread((write_idx + 1+line) % LOG_STRUCT_MESSAGE_LINES);
-      if (timeStamp[tmpread] != 0) {
-        output += formatLine(tmpread, lineEnd);
-      }
-      return !isEmpty();
-    }
-
-    bool getAll(String& output, const String& lineEnd) {
-      int tmpread((write_idx + 1) % LOG_STRUCT_MESSAGE_LINES);
-      bool someAdded = false;
-      while (tmpread != write_idx) {
-        if (timeStamp[tmpread] != 0) {
-          output += formatLine(tmpread, lineEnd);
-          someAdded = true;
-        }
-        tmpread = (tmpread + 1)% LOG_STRUCT_MESSAGE_LINES;
-      }
-      return someAdded;
-    }
-
     bool isEmpty() {
       return (write_idx == read_idx);
     }
 
+    bool logActiveRead() {
+      clearExpiredEntries();
+      return timePassedSince(lastReadTimeStamp) < LOG_BUFFER_EXPIRE;
+    }
+
   private:
     String formatLine(int index, const String& lineEnd) {
       String output;
@@ -1167,26 +1316,50 @@ struct LogStruct {
       return output;
     }
 
+    void clearExpiredEntries() {
+      if (isEmpty()) {
+        return;
+      }
+      if (timePassedSince(lastReadTimeStamp) > LOG_BUFFER_EXPIRE) {
+        // Clear the entire log.
+        // If web log is the only log active, it will not be checked again until it is read.
+        for (read_idx = 0; read_idx < LOG_STRUCT_MESSAGE_LINES; ++read_idx) {
+          Message[read_idx] = String(); // Free also the reserved memory.
+          timeStamp[read_idx] = 0;
+          log_level[read_idx] = 0;
+        }
+        read_idx = 0;
+        write_idx = 0;
+      }
+    }
 
     int write_idx;
     int read_idx;
     unsigned long timeStamp[LOG_STRUCT_MESSAGE_LINES];
+    unsigned long lastReadTimeStamp;
     byte log_level[LOG_STRUCT_MESSAGE_LINES];
     String Message[LOG_STRUCT_MESSAGE_LINES];
 
 } Logging;
 
+std::deque<char> serialLogBuffer;
+unsigned long last_serial_log_read = 0;
+
 byte highest_active_log_level = 0;
 bool log_to_serial_disabled = false;
+bool log_to_serial_disabled_temporary = false;
 // Do this in a template to prevent casting to String when not needed.
 #define addLog(L,S) if (loglevelActiveFor(L)) { addToLog(L,S); }
 
+/*********************************************************************************************\
+ * DeviceStruct
+\*********************************************************************************************/
 struct DeviceStruct
 {
   DeviceStruct() :
-    Number(0), Type(0), VType(0), Ports(0),
+    Number(0), Type(0), VType(0), Ports(0), ValueCount(0),
     PullUpOption(false), InverseLogicOption(false), FormulaOption(false),
-    ValueCount(0), Custom(false), SendDataOption(false), GlobalSyncOption(false),
+    Custom(false), SendDataOption(false), GlobalSyncOption(false),
     TimerOption(false), TimerOptional(false), DecimalsOnly(false) {}
 
   bool connectedToGPIOpins() {
@@ -1198,33 +1371,45 @@ struct DeviceStruct
   byte Type;    // How the device is connected. e.g. DEVICE_TYPE_SINGLE => connected through 1 datapin
   byte VType;   // Type of value the plugin will return, used only for Domoticz
   byte Ports;   // Port to use when device has multiple I/O pins  (N.B. not used much)
+  byte ValueCount;            // The number of output values of a plugin. The value should match the number of keys PLUGIN_VALUENAME1_xxx
   boolean PullUpOption;       // Allow to set internal pull-up resistors.
   boolean InverseLogicOption; // Allow to invert the boolean state (e.g. a switch)
   boolean FormulaOption;      // Allow to enter a formula to convert values during read. (not possible with Custom enabled)
-  byte ValueCount;            // The number of output values of a plugin. The value should match the number of keys PLUGIN_VALUENAME1_xxx
   boolean Custom;
   boolean SendDataOption;     // Allow to send data to a controller.
   boolean GlobalSyncOption;   // No longer used. Was used for ESPeasy values sync between nodes
   boolean TimerOption;        // Allow to set the "Interval" timer for the plugin.
   boolean TimerOptional;      // When taskdevice timer is not set and not optional, use default "Interval" delay (Settings.Delay)
   boolean DecimalsOnly;       // Allow to set the number of decimals (otherwise treated a 0 decimals)
-} Device[DEVICES_MAX + 1]; // 1 more because first device is empty device
+};
+typedef std::vector<DeviceStruct> DeviceVector;
+DeviceVector Device;
 
+
+/*********************************************************************************************\
+ * ProtocolStruct
+\*********************************************************************************************/
 struct ProtocolStruct
 {
   ProtocolStruct() :
-    Number(0), usesMQTT(false), usesAccount(false), usesPassword(false),
-    defaultPort(0), usesTemplate(false), usesID(false), Custom(false) {}
+    defaultPort(0), Number(0), usesMQTT(false), usesAccount(false), usesPassword(false),
+    usesTemplate(false), usesID(false), Custom(false) {}
+  uint16_t defaultPort;
   byte Number;
   boolean usesMQTT;
   boolean usesAccount;
   boolean usesPassword;
-  int defaultPort;
   boolean usesTemplate;
   boolean usesID;
   boolean Custom;
-} Protocol[CPLUGIN_MAX];
+};
+typedef std::vector<ProtocolStruct> ProtocolVector;
+ProtocolVector Protocol;
 
+
+/*********************************************************************************************\
+ * NotificationStruct
+\*********************************************************************************************/
 struct NotificationStruct
 {
   NotificationStruct() :
@@ -1234,20 +1419,29 @@ struct NotificationStruct
   byte usesGPIO;
 } Notification[NPLUGIN_MAX];
 
+
+/*********************************************************************************************\
+ * NodeStruct
+\*********************************************************************************************/
 struct NodeStruct
 {
   NodeStruct() :
-    age(0), build(0), nodeName(NULL), nodeType(0)
+    build(0), age(0), nodeType(0)
     {
       for (byte i = 0; i < 4; ++i) ip[i] = 0;
     }
+  String nodeName;
   byte ip[4];
-  byte age;
   uint16_t build;
-  char* nodeName;
+  byte age;
   byte nodeType;
-} Nodes[UNIT_MAX];
+};
+typedef std::map<byte, NodeStruct> NodesMap;
+NodesMap Nodes;
 
+/*********************************************************************************************\
+ * systemTimerStruct
+\*********************************************************************************************/
 struct systemTimerStruct
 {
   systemTimerStruct() :
@@ -1264,8 +1458,12 @@ struct systemTimerStruct
 };
 std::map<unsigned long, systemTimerStruct> systemTimers;
 
+/*********************************************************************************************\
+ * pinStatesStruct
+\*********************************************************************************************/
 struct pinStatesStruct
 {
+  pinStatesStruct() : plugin(0), index(0), mode(0), value(0) {}
   byte plugin;
   byte index;
   byte mode;
@@ -1277,21 +1475,26 @@ struct pinStatesStruct
 #define RTC_BASE_STRUCT 64
 #define RTC_BASE_USERVAR 74
 
+/*********************************************************************************************\
+ * RTCStruct
+\*********************************************************************************************/
 //max 40 bytes: ( 74 - 64 ) * 4
 struct RTCStruct
 {
+  RTCStruct() : ID1(0), ID2(0), unused1(false), factoryResetCounter(0),
+                deepSleepState(0), bootFailedCount(0), flashDayCounter(0),
+                flashCounter(0), bootCounter(0) {}
   byte ID1;
   byte ID2;
   boolean unused1;
   byte factoryResetCounter;
   byte deepSleepState;
-  byte unused2;
+  byte bootFailedCount;
   byte flashDayCounter;
   unsigned long flashCounter;
   unsigned long bootCounter;
 } RTC;
 
-
 int deviceCount = -1;
 int protocolCount = -1;
 int notificationCount = -1;
@@ -1301,8 +1504,14 @@ String printWebString = "";
 boolean printToWebJSON = false;
 
 float UserVar[VARS_PER_TASK * TASKS_MAX];
+
+/*********************************************************************************************\
+ * rulesTimerStruct
+\*********************************************************************************************/
 struct rulesTimerStatus
 {
+  rulesTimerStatus() : timestamp(0), interval(0), paused(false) {}
+
   unsigned long timestamp;
   unsigned int interval; //interval in milliseconds
   boolean paused;
@@ -1310,14 +1519,15 @@ struct rulesTimerStatus
 
 msecTimerHandlerStruct msecTimerHandler;
 
-unsigned long timermqtt_interval;
-unsigned long lastSend;
-unsigned long lastWeb;
+unsigned long timermqtt_interval = 250;
+unsigned long lastSend = 0;
+unsigned long lastWeb = 0;
 byte cmd_within_mainloop = 0;
-unsigned long connectionFailures;
+unsigned long connectionFailures = 0;
 unsigned long wdcounter = 0;
 unsigned long timerAPoff = 0;
 unsigned long timerAwakeFromDeepSleep = 0;
+unsigned long last_system_event_run = 0;
 
 #if FEATURE_ADC_VCC
 float vcc = -1.0;
@@ -1464,16 +1674,22 @@ String eventBuffer = "";
 
 uint32_t lowestRAM = 0;
 String lowestRAMfunction = "";
+uint32_t lowestFreeStack = 0;
+String lowestFreeStackfunction = "";
+
 
 bool shouldReboot=false;
 bool firstLoop=true;
 
 boolean activeRuleSets[RULESETS_MAX];
 
-boolean       UseRTOSMultitasking;
+boolean UseRTOSMultitasking = false;
 
-void (*MainLoopCall_ptr)(void);
+// void (*MainLoopCall_ptr)(void); //FIXME TD-er: No idea what this does.
 
+/*********************************************************************************************\
+ * TimingStats
+\*********************************************************************************************/
 class TimingStats {
     public:
       TimingStats() : _timeTotal(0.0), _count(0), _maxVal(0), _minVal(4294967295) {}
@@ -1619,6 +1835,21 @@ unsigned long timediff_cpu_cycles_total = 0;
 #define PROC_SYS_TIMER        9
 #define SET_NEW_TIMER        10
 #define TIME_DIFF_COMPUTE    11
+#define MQTT_DELAY_QUEUE     12
+#define C001_DELAY_QUEUE     13
+#define C002_DELAY_QUEUE     14
+#define C003_DELAY_QUEUE     15
+#define C004_DELAY_QUEUE     16
+#define C005_DELAY_QUEUE     17
+#define C006_DELAY_QUEUE     18
+#define C007_DELAY_QUEUE     19
+#define C008_DELAY_QUEUE     20
+#define C009_DELAY_QUEUE     21
+#define C010_DELAY_QUEUE     22
+#define C011_DELAY_QUEUE     23
+#define C012_DELAY_QUEUE     24
+#define C013_DELAY_QUEUE     25
+
 
 
 
@@ -1644,6 +1875,25 @@ String getMiscStatsName(int stat) {
         case PROC_SYS_TIMER:        return F("proc_system_timer() ");
         case SET_NEW_TIMER:         return F("setNewTimerAt()     ");
         case TIME_DIFF_COMPUTE:     return F("timeDiff()          ");
+        case MQTT_DELAY_QUEUE:      return F("Delay queue     MQTT");
+        case C001_DELAY_QUEUE:
+        case C002_DELAY_QUEUE:
+        case C003_DELAY_QUEUE:
+        case C004_DELAY_QUEUE:
+        case C005_DELAY_QUEUE:
+        case C006_DELAY_QUEUE:
+        case C007_DELAY_QUEUE:
+        case C008_DELAY_QUEUE:
+        case C009_DELAY_QUEUE:
+        case C010_DELAY_QUEUE:
+        case C011_DELAY_QUEUE:
+        case C012_DELAY_QUEUE:
+        case C013_DELAY_QUEUE:
+        {
+          String result = F("Delay queue     ");
+          result += get_formatted_Controller_number(static_cast<int>(stat - C001_DELAY_QUEUE + 1));
+          return result;
+        }
     }
     return F("Unknown");
 }

src/ESPEasy.ino

@@ -83,6 +83,8 @@
 // Define globals before plugin sets to allow a personal override of the selected plugins
 #include "ESPEasy-Globals.h"
 #include "define_plugin_sets.h"
+// Plugin helper needs the defined controller sets, thus include after 'define_plugin_sets.h'
+#include "_CPlugin_Helper.h"
 
 // Blynk_get prototype
 boolean Blynk_get(const String& command, byte controllerIndex,float *data = NULL );
@@ -104,9 +106,14 @@ int firstEnabledBlynkController() {
 \*********************************************************************************************/
 void setup()
 {
+#ifdef ESP8266_DISABLE_EXTRA4K
+  disable_extra4k_at_link_time();
+#endif
   WiFi.persistent(false); // Do not use SDK storage of SSID/WPA parameters
   WiFi.setAutoReconnect(false);
   setWifiMode(WIFI_OFF);
+  lowestFreeStack = getFreeStackWatermark();
+  lowestRAM = FreeMem();
 
   Plugin_id.resize(PLUGIN_MAX);
   Task_id_to_Plugin_id.resize(TASKS_MAX);
@@ -117,8 +124,6 @@ void setup()
       ledChannelPin[x] = -1;
   #endif
 
-  lowestRAM = FreeMem();
-
   Serial.begin(115200);
   // Serial.print("\n\n\nBOOOTTT\n\n\n");
 
@@ -156,6 +161,7 @@ void setup()
   //warm boot
   if (readFromRTC())
   {
+    RTC.bootFailedCount++;
     RTC.bootCounter++;
     readUserVarFromRTC();
 
@@ -191,6 +197,16 @@ void setup()
   fileSystemCheck();
   progMemMD5check();
   LoadSettings();
+  if (RTC.bootFailedCount > 10 && RTC.bootCounter > 10) {
+    byte toDisable = RTC.bootFailedCount - 10;
+    toDisable = disablePlugin(toDisable);
+    if (toDisable != 0) {
+      toDisable = disableController(toDisable);
+    }
+    if (toDisable != 0) {
+      toDisable = disableNotification(toDisable);
+    }
+  }
 
 //  setWifiMode(WIFI_STA);
   checkRuleSets();
@@ -302,9 +318,17 @@ void setup()
     if(UseRTOSMultitasking){
       log = F("RTOS : Launching tasks");
       addLog(LOG_LEVEL_INFO, log);
-      xTaskCreatePinnedToCore(RTOS_TaskServers, "RTOS_TaskServers", 8192, NULL, 1, NULL, 1);
+      xTaskCreatePinnedToCore(RTOS_TaskServers, "RTOS_TaskServers", 16384, NULL, 1, NULL, 1);
       xTaskCreatePinnedToCore(RTOS_TaskSerial, "RTOS_TaskSerial", 8192, NULL, 1, NULL, 1);
       xTaskCreatePinnedToCore(RTOS_Task10ps, "RTOS_Task10ps", 8192, NULL, 1, NULL, 1);
+      xTaskCreatePinnedToCore(
+                    RTOS_HandleSchedule,   /* Function to implement the task */
+                    "RTOS_HandleSchedule", /* Name of the task */
+                    16384,      /* Stack size in words */
+                    NULL,       /* Task input parameter */
+                    1,          /* Priority of the task */
+                    NULL,       /* Task handle. */
+                    1);         /* Core where the task should run */
     }
   #endif
 
@@ -351,6 +375,14 @@ void RTOS_Task10ps( void * parameter )
     run10TimesPerSecond();
  }
 }
+
+void RTOS_HandleSchedule( void * parameter )
+{
+ while (true){
+    handle_schedule();
+ }
+}
+
 #endif
 
 int firstEnabledMQTTController() {
@@ -433,8 +465,11 @@ int getLoopCountPerSec() {
 \*********************************************************************************************/
 void loop()
 {
+  /*
+  //FIXME TD-er: No idea what this does.
   if(MainLoopCall_ptr)
       MainLoopCall_ptr();
+  */
 
   updateLoopStats();
 
@@ -462,8 +497,11 @@ void loop()
   if (firstLoopConnectionsEstablished) {
      firstLoop = false;
      timerAwakeFromDeepSleep = millis(); // Allow to run for "awake" number of seconds, now we have wifi.
-     schedule_all_task_device_timers();
-   }
+     // schedule_all_task_device_timers(); Disabled for now, since we are now using queues for controllers.
+
+     RTC.bootFailedCount = 0;
+     saveToRTC();
+  }
 
   // Deep sleep mode, just run all tasks one (more) time and go back to sleep as fast as possible
   if ((firstLoopConnectionsEstablished || readyForSleep()) && isDeepSleepEnabled())
@@ -478,7 +516,10 @@ void loop()
   //normal mode, run each task when its time
   else
   {
-    handle_schedule();
+    if (!UseRTOSMultitasking) {
+      // On ESP32 the schedule is executed on the 2nd core.
+      handle_schedule();
+    }
   }
 
   backgroundtasks();
@@ -529,11 +570,34 @@ void runPeriodicalMQTT() {
   }
 }
 
+String getMQTT_state() {
+  switch (MQTTclient.state()) {
+    case MQTT_CONNECTION_TIMEOUT     : return F("Connection timeout");
+    case MQTT_CONNECTION_LOST        : return F("Connection lost");
+    case MQTT_CONNECT_FAILED         : return F("Connect failed");
+    case MQTT_DISCONNECTED           : return F("Disconnected");
+    case MQTT_CONNECTED              : return F("Connected");
+    case MQTT_CONNECT_BAD_PROTOCOL   : return F("Connect bad protocol");
+    case MQTT_CONNECT_BAD_CLIENT_ID  : return F("Connect bad client_id");
+    case MQTT_CONNECT_UNAVAILABLE    : return F("Connect unavailable");
+    case MQTT_CONNECT_BAD_CREDENTIALS: return F("Connect bad credentials");
+    case MQTT_CONNECT_UNAUTHORIZED   : return F("Connect unauthorized");
+    default: return "";
+  }
+}
+
 void updateMQTTclient_connected() {
   if (MQTTclient_connected != MQTTclient.connected()) {
     MQTTclient_connected = !MQTTclient_connected;
-    if (!MQTTclient_connected)
-      addLog(LOG_LEVEL_ERROR, F("MQTT : Connection lost"));
+    if (!MQTTclient_connected) {
+      if (loglevelActiveFor(LOG_LEVEL_ERROR)) {
+        String connectionError = F("MQTT : Connection lost, state: ");
+        connectionError += getMQTT_state();
+        addLog(LOG_LEVEL_ERROR, connectionError);
+      }
+    } else {
+      schedule_all_tasks_using_MQTT_controller();
+    }
     if (Settings.UseRules) {
       String event = MQTTclient_connected ? F("MQTT#Connected") : F("MQTT#Disconnected");
       rulesProcessing(event);
@@ -775,13 +839,10 @@ void SensorSendTask(byte TaskIndex)
       {
         if (ExtraTaskSettings.TaskDeviceFormula[varNr][0] != 0)
         {
-          String spreValue = String(preValue[varNr]);
           String formula = ExtraTaskSettings.TaskDeviceFormula[varNr];
-          float value = UserVar[varIndex + varNr];
+          formula.replace(F("%pvalue%"), String(preValue[varNr]));
+          formula.replace(F("%value%"), String(UserVar[varIndex + varNr]));
           float result = 0;
-          String svalue = String(value);
-          formula.replace(F("%pvalue%"), spreValue);
-          formula.replace(F("%value%"), svalue);
           byte error = Calculate(formula.c_str(), &result);
           if (error == 0)
             UserVar[varIndex + varNr] = result;
@@ -803,6 +864,16 @@ void backgroundtasks()
   //checkRAM(F("backgroundtasks"));
   //always start with a yield
   yield();
+/*
+  // Remove this watchdog feed for now.
+  // See https://github.com/letscontrolit/ESPEasy/issues/1722#issuecomment-419659193
+
+  #ifdef ESP32
+  // Have to find a similar function to call ESP32's esp_task_wdt_feed();
+  #else
+  ESP.wdtFeed();
+  #endif
+*/
 
   //prevent recursion!
   if (runningBackgroundTasks)
@@ -814,7 +885,7 @@ void backgroundtasks()
   #if defined(ESP8266)
     tcpCleanup();
   #endif
-
+  process_serialLogBuffer();
   if(!UseRTOSMultitasking){
     if (Settings.UseSerial)
       if (Serial.available())

src/ESPEasyStorage.ino

@@ -77,7 +77,7 @@ String BuildFixes()
     Settings.UseRTOSMultitasking = false;
     Settings.Pin_Reset = -1;
     Settings.SyslogFacility = DEFAULT_SYSLOG_FACILITY;
-    Settings.MQTTUseUnitNameAsClientId = DEFAULT_MQTT_USE_UNITNANE_AS_CLIENTID;
+    Settings.MQTTUseUnitNameAsClientId = DEFAULT_MQTT_USE_UNITNAME_AS_CLIENTID;
     Settings.StructSize = sizeof(Settings);
   }
 
@@ -113,7 +113,7 @@ void fileSystemCheck()
     {
       ResetFactory();
     }
-    f.close();
+    if (f) f.close();
   }
   else
   {
@@ -147,9 +147,14 @@ String SaveSettings(void)
     // Settings have changed, save to file.
     memcpy(Settings.md5, tmp_md5, 16);
 */
+    Settings.validate();
     err=SaveToFile((char*)FILE_CONFIG, 0, (byte*)&Settings, sizeof(Settings));
     if (err.length())
-     return(err);
+      return(err);
+    // Must check this after saving, or else it is not possible to fix multiple
+    // issues which can only corrected on different pages.
+    if (!SettingsCheck(err)) return err;
+
 //  }
 
   memcpy( SecuritySettings.ProgmemMd5, CRCValues.runTimeMD5, 16);
@@ -182,6 +187,7 @@ String LoadSettings()
   err=LoadFromFile((char*)FILE_CONFIG, 0, (byte*)&Settings, sizeof( SettingsStruct));
   if (err.length())
     return(err);
+  Settings.validate();
 
     // FIXME @TD-er: As discussed in #1292, the CRC for the settings is now disabled.
 /*
@@ -219,6 +225,51 @@ String LoadSettings()
   return(err);
 }
 
+/********************************************************************************************\
+  Disable Plugin, based on bootFailedCount
+  \*********************************************************************************************/
+byte disablePlugin(byte bootFailedCount) {
+  for (byte i = 0; i < TASKS_MAX && bootFailedCount > 0; ++i) {
+    if (Settings.TaskDeviceEnabled[i]) {
+      --bootFailedCount;
+      if (bootFailedCount == 0) {
+        Settings.TaskDeviceEnabled[i] = false;
+      }
+    }
+  }
+  return bootFailedCount;
+}
+
+/********************************************************************************************\
+  Disable Controller, based on bootFailedCount
+  \*********************************************************************************************/
+byte disableController(byte bootFailedCount) {
+  for (byte i = 0; i < CONTROLLER_MAX && bootFailedCount > 0; ++i) {
+    if (Settings.ControllerEnabled[i]) {
+      --bootFailedCount;
+      if (bootFailedCount == 0) {
+        Settings.ControllerEnabled[i] = false;
+      }
+    }
+  }
+  return bootFailedCount;
+}
+
+/********************************************************************************************\
+  Disable Notification, based on bootFailedCount
+  \*********************************************************************************************/
+byte disableNotification(byte bootFailedCount) {
+  for (byte i = 0; i < NOTIFICATION_MAX && bootFailedCount > 0; ++i) {
+    if (Settings.NotificationEnabled[i]) {
+      --bootFailedCount;
+      if (bootFailedCount == 0) {
+        Settings.NotificationEnabled[i] = false;
+      }
+    }
+  }
+  return bootFailedCount;
+}
+
 /********************************************************************************************\
   Offsets in settings files
   \*********************************************************************************************/
@@ -388,20 +439,25 @@ String LoadCustomTaskSettings(int TaskIndex, byte* memAddress, int datasize)
 /********************************************************************************************\
   Save Controller settings to SPIFFS
   \*********************************************************************************************/
-String SaveControllerSettings(int ControllerIndex, byte* memAddress, int datasize)
+String SaveControllerSettings(int ControllerIndex, ControllerSettingsStruct& controller_settings)
 {
   checkRAM(F("SaveControllerSettings"));
-  return SaveToFile(ControllerSettings_Type, ControllerIndex, (char*)FILE_CONFIG, memAddress, datasize);
+  controller_settings.validate(); // Make sure the saved controller settings have proper values.
+  return SaveToFile(ControllerSettings_Type, ControllerIndex,
+                    (char*)FILE_CONFIG, (byte*)&controller_settings, sizeof(controller_settings));
 }
 
 
 /********************************************************************************************\
   Load Controller settings to SPIFFS
   \*********************************************************************************************/
-String LoadControllerSettings(int ControllerIndex, byte* memAddress, int datasize)
-{
+String LoadControllerSettings(int ControllerIndex, ControllerSettingsStruct& controller_settings) {
   checkRAM(F("LoadControllerSettings"));
-  return(LoadFromFile(ControllerSettings_Type, ControllerIndex, (char*)FILE_CONFIG, memAddress, datasize));
+  String result =
+    LoadFromFile(ControllerSettings_Type, ControllerIndex,
+                 (char*)FILE_CONFIG, (byte*)&controller_settings, sizeof(controller_settings));
+  controller_settings.validate(); // Make sure the loaded controller settings have proper values.
+  return result;
 }
 
 
@@ -466,13 +522,15 @@ String InitFile(const char* fname, int datasize)
   FLASH_GUARD();
 
   fs::File f = SPIFFS.open(fname, "w");
-  SPIFFS_CHECK(f, fname);
+  if (f) {
+    SPIFFS_CHECK(f, fname);
 
-  for (int x = 0; x < datasize ; x++)
-  {
-    SPIFFS_CHECK(f.write(0), fname);
+    for (int x = 0; x < datasize ; x++)
+    {
+      SPIFFS_CHECK(f.write(0), fname);
+    }
+    f.close();
   }
-  f.close();
 
   //OK
   return String();
@@ -483,6 +541,15 @@ String InitFile(const char* fname, int datasize)
   \*********************************************************************************************/
 String SaveToFile(char* fname, int index, byte* memAddress, int datasize)
 {
+#ifndef ESP32
+  if (allocatedOnStack(memAddress)) {
+    String log = F("SaveToFile: ");
+    log += fname;
+    log += F(" ERROR, Data allocated on stack");
+    addLog(LOG_LEVEL_ERROR, log);
+//    return log;  // FIXME TD-er: Should this be considered a breaking error?
+  }
+#endif
   if (index < 0) {
     String log = F("SaveToFile: ");
     log += fname;
@@ -495,19 +562,27 @@ String SaveToFile(char* fname, int index, byte* memAddress, int datasize)
   FLASH_GUARD();
 
   fs::File f = SPIFFS.open(fname, "r+");
-  SPIFFS_CHECK(f, fname);
+  if (f) {
+    SPIFFS_CHECK(f, fname);
 
-  SPIFFS_CHECK(f.seek(index, fs::SeekSet), fname);
-  byte *pointerToByteToSave = memAddress;
-  for (int x = 0; x < datasize ; x++)
-  {
-    SPIFFS_CHECK(f.write(*pointerToByteToSave), fname);
-    pointerToByteToSave++;
+    SPIFFS_CHECK(f.seek(index, fs::SeekSet), fname);
+    byte *pointerToByteToSave = memAddress;
+    for (int x = 0; x < datasize ; x++)
+    {
+      SPIFFS_CHECK(f.write(*pointerToByteToSave), fname);
+      pointerToByteToSave++;
+    }
+    f.close();
+    String log = F("FILE : Saved ");
+    log=log+fname;
+    addLog(LOG_LEVEL_INFO, log);
+  } else {
+    String log = F("SaveToFile: ");
+    log += fname;
+    log += F(" ERROR, Cannot save to file");
+    addLog(LOG_LEVEL_ERROR, log);
+    return log;
   }
-  f.close();
-  String log = F("FILE : Saved ");
-  log=log+fname;
-  addLog(LOG_LEVEL_INFO, log);
 
   //OK
   return String();
@@ -530,14 +605,22 @@ String ClearInFile(char* fname, int index, int datasize)
   FLASH_GUARD();
 
   fs::File f = SPIFFS.open(fname, "r+");
-  SPIFFS_CHECK(f, fname);
+  if (f) {
+    SPIFFS_CHECK(f, fname);
 
-  SPIFFS_CHECK(f.seek(index, fs::SeekSet), fname);
-  for (int x = 0; x < datasize ; x++)
-  {
-    SPIFFS_CHECK(f.write(0), fname);
+    SPIFFS_CHECK(f.seek(index, fs::SeekSet), fname);
+    for (int x = 0; x < datasize ; x++)
+    {
+      SPIFFS_CHECK(f.write(0), fname);
+    }
+    f.close();
+  } else {
+    String log = F("ClearInFile: ");
+    log += fname;
+    log += F(" ERROR, Cannot save to file");
+    addLog(LOG_LEVEL_ERROR, log);
+    return log;
   }
-  f.close();
 
   //OK
   return String();

src/ESPEasyWifi.ino

@@ -126,6 +126,7 @@ void processGotIP() {
     initTime();
   }
   mqtt_reconnect_count = 0;
+  MQTTclient_should_reconnect = true;
   timermqtt_interval = 100;
   setIntervalTimer(TIMER_MQTT);
   if (Settings.UseRules)
@@ -417,8 +418,10 @@ void setWifiMode(WiFiMode_t wifimode) {
 String WifiGetAPssid()
 {
   String ssid(Settings.Name);
-  ssid+=F("_");
-  ssid+=Settings.Unit;
+  if (Settings.appendUnitToHostname()) {
+    ssid+=F("_");
+    ssid+=Settings.Unit;
+  }
   return (ssid);
 }
 
@@ -675,6 +678,7 @@ void WifiScan()
 
 String formatScanResult(int i, const String& separator) {
   String result = WiFi.SSID(i);
+  htmlEscape(result);
   #ifndef ESP32
   if (WiFi.isHidden(i)) {
     result += F("#Hidden#");

src/I2C.ino

@@ -11,6 +11,12 @@ bool I2C_read_words(uint8_t i2caddr, I2Cdata_words& data) {
   return size == i2cdev.readWords(i2caddr, data.getRegister(), size, data.get());
 }
 
+// See https://github.com/platformio/platform-espressif32/issues/126
+#ifdef ESP32
+  #define END_TRANSMISSION_FLAG true
+#else
+  #define END_TRANSMISSION_FLAG false
+#endif
 
 //**************************************************************************/
 // Wake up I2C device
@@ -66,7 +72,7 @@ uint8_t I2C_read8_reg(uint8_t i2caddr, byte reg, bool * is_ok) {
 
   Wire.beginTransmission(i2caddr);
   Wire.write((uint8_t)reg);
-  Wire.endTransmission(false);
+  Wire.endTransmission(END_TRANSMISSION_FLAG);
   byte count = Wire.requestFrom(i2caddr, (byte)1);
   if (is_ok != NULL) {
     *is_ok = (count == 1);
@@ -84,7 +90,7 @@ uint16_t I2C_read16_reg(uint8_t i2caddr, byte reg) {
 
   Wire.beginTransmission(i2caddr);
   Wire.write((uint8_t)reg);
-  Wire.endTransmission(false);
+  Wire.endTransmission(END_TRANSMISSION_FLAG);
   Wire.requestFrom(i2caddr, (byte)2);
   value = (Wire.read() << 8) | Wire.read();
 
@@ -99,7 +105,7 @@ int32_t I2C_read24_reg(uint8_t i2caddr, byte reg) {
 
   Wire.beginTransmission(i2caddr);
   Wire.write((uint8_t)reg);
-  Wire.endTransmission(false);
+  Wire.endTransmission(END_TRANSMISSION_FLAG);
   Wire.requestFrom(i2caddr, (byte)3);
   value = (((int32_t)Wire.read()) << 16) | (Wire.read() << 8) | Wire.read();
 
@@ -114,8 +120,8 @@ int32_t I2C_read32_reg(uint8_t i2caddr, byte reg) {
 
   Wire.beginTransmission(i2caddr);
   Wire.write((uint8_t)reg);
-  Wire.endTransmission(false);
-  Wire.requestFrom(i2caddr, (byte)3);
+  Wire.endTransmission(END_TRANSMISSION_FLAG);
+  Wire.requestFrom(i2caddr, (byte)4);
   value = (((int32_t)Wire.read()) <<24) | (((uint32_t)Wire.read()) << 16) | (Wire.read() << 8) | Wire.read();
 
   return value;
@@ -139,3 +145,5 @@ int16_t I2C_readS16_reg(uint8_t i2caddr, byte reg) {
 int16_t I2C_readS16_LE_reg(uint8_t i2caddr, byte reg) {
   return (int16_t)I2C_read16_LE_reg(i2caddr, reg);
 }
+
+#undef END_TRANSMISSION_FLAG

src/Networking.ino

@@ -107,20 +107,21 @@ void checkUDP()
                 for (byte x = 0; x < 4; x++)
                   ip[x] = packetBuffer[x + 8];
 
-                if (unit < UNIT_MAX)
-                {
+                Nodes[unit].age = 0; // Create a new element when not present
+                NodesMap::iterator it = Nodes.find(unit);
+                if (it != Nodes.end()) {
                   for (byte x = 0; x < 4; x++)
-                    Nodes[unit].ip[x] = packetBuffer[x + 8];
-                  Nodes[unit].age = 0; // reset 'age counter'
+                    it->second.ip[x] = packetBuffer[x + 8];
+                  it->second.age = 0; // reset 'age counter'
                   if (len >= 41) // extended packet size
                   {
-                    // FIXME TD-er: Memory leak waiting to happen.
-                    Nodes[unit].build = packetBuffer[13] + 256*packetBuffer[14];
-                    if (Nodes[unit].nodeName==0)
-                        Nodes[unit].nodeName =  (char *)malloc(26);
-                    memcpy(Nodes[unit].nodeName,reinterpret_cast<byte*>(&packetBuffer[15]), 25);
-                    Nodes[unit].nodeName[25]=0;
-                    Nodes[unit].nodeType = packetBuffer[40];
+                    it->second.build = packetBuffer[13] + 256*packetBuffer[14];
+                    char tmpNodeName[26] = {0};
+                    memcpy(&tmpNodeName[0], reinterpret_cast<byte*>(&packetBuffer[15]), 25);
+                    tmpNodeName[25] = 0;
+                    it->second.nodeName = tmpNodeName;
+                    it->second.nodeName.trim();
+                    it->second.nodeType = packetBuffer[40];
                   }
                 }
 
@@ -166,17 +167,17 @@ void SendUDPCommand(byte destUnit, char* data, byte dataLength)
   if (!WiFiConnected(100)) {
     return;
   }
-  byte firstUnit = 1;
-  byte lastUnit = UNIT_MAX - 1;
   if (destUnit != 0)
   {
-    firstUnit = destUnit;
-    lastUnit = destUnit;
-  }
-  for (int x = firstUnit; x <= lastUnit; x++)
-  {
-    sendUDP(x, (byte*)data, dataLength);
+    sendUDP(destUnit, (byte*)data, dataLength);
     delay(10);
+  } else {
+    for (NodesMap::iterator it = Nodes.begin(); it != Nodes.end(); ++it) {
+      if (it->first != Settings.Unit) {
+        sendUDP(it->first, (byte*)data, dataLength);
+        delay(10);
+      }
+    }
   }
   delay(50);
 }
@@ -190,9 +191,18 @@ void sendUDP(byte unit, byte* data, byte size)
   if (!WiFiConnected(100)) {
     return;
   }
-  if (unit != 255)
-    if (Nodes[unit].ip[0] == 0)
+
+  IPAddress remoteNodeIP;
+  if (unit == 255)
+    remoteNodeIP = {255,255,255,255};
+  else {
+    NodesMap::iterator it = Nodes.find(unit);
+    if (it == Nodes.end())
       return;
+    if (it->second.ip[0] == 0)
+      return;
+    remoteNodeIP = it->second.ip;
+  }
 
   if (loglevelActiveFor(LOG_LEVEL_DEBUG_MORE)) {
     String log = F("UDP  : Send UDP message to ");
@@ -201,12 +211,6 @@ void sendUDP(byte unit, byte* data, byte size)
   }
 
   statusLED(true);
-
-  IPAddress remoteNodeIP;
-  if (unit == 255)
-    remoteNodeIP = {255,255,255,255};
-  else
-    remoteNodeIP = Nodes[unit].ip;
   portUDP.beginPacket(remoteNodeIP, Settings.UDPPort);
   portUDP.write(data, size);
   portUDP.endPacket();
@@ -218,14 +222,17 @@ void sendUDP(byte unit, byte* data, byte size)
   \*********************************************************************************************/
 void refreshNodeList()
 {
-  for (byte counter = 0; counter < UNIT_MAX; counter++)
-  {
-    if (Nodes[counter].ip[0] != 0)
-    {
-      Nodes[counter].age++;  // increment age counter
-      if (Nodes[counter].age > 10) // if entry to old, clear this node ip from the list.
-        for (byte x = 0; x < 4; x++)
-          Nodes[counter].ip[x] = 0;
+  for (NodesMap::iterator it = Nodes.begin(); it != Nodes.end(); ) {
+    bool mustRemove = true;
+    if (it->second.ip[0] != 0) {
+      if (it->second.age < 10) {
+        it->second.age++;
+        mustRemove = false;
+        ++it;
+      }
+    }
+    if (mustRemove) {
+      it = Nodes.erase(it);
     }
   }
 }
@@ -277,15 +284,17 @@ void sendSysInfoUDP(byte repeats)
       delay(500);
   }
 
-  // store my own info also in the list...
-  if (Settings.Unit < UNIT_MAX)
+  Nodes[Settings.Unit].age = 0; // Create new node when not already present.
+  // store my own info also in the list
+  NodesMap::iterator it = Nodes.find(Settings.Unit);
+  if (it != Nodes.end())
   {
     IPAddress ip = WiFi.localIP();
     for (byte x = 0; x < 4; x++)
-      Nodes[Settings.Unit].ip[x] = ip[x];
-    Nodes[Settings.Unit].age = 0;
-    Nodes[Settings.Unit].build = Settings.Build;
-    Nodes[Settings.Unit].nodeType = NODE_TYPE_ID;
+      it->second.ip[x] = ip[x];
+    it->second.age = 0;
+    it->second.build = Settings.Build;
+    it->second.nodeType = NODE_TYPE_ID;
   }
 }
 

src/Scheduler.ino

@@ -41,18 +41,26 @@ unsigned long getMixedId(unsigned long timerType, unsigned long id) {
 \*********************************************************************************************/
 void handle_schedule() {
   unsigned long timer;
-  const unsigned long mixed_id = msecTimerHandler.getNextId(timer);
+  unsigned long mixed_id = 0;
+  if (timePassedSince(last_system_event_run) < 500) {
+    // Make sure system event queue will be looked at every now and then.
+    mixed_id = msecTimerHandler.getNextId(timer);
+  }
   if (mixed_id == 0) {
     // No id ready to run right now.
     // Events are not that important to run immediately.
     // Make sure normal scheduled jobs run at higher priority.
+    backgroundtasks();
     process_system_event_queue();
+    last_system_event_run = millis();
     return;
   }
   const unsigned long timerType = (mixed_id >> TIMER_ID_SHIFT);
   const unsigned long mask = (1 << TIMER_ID_SHIFT) -1;
   const unsigned long id = mixed_id & mask;
 
+  yield(); // See: https://github.com/letscontrolit/ESPEasy/issues/1818#issuecomment-425351328
+
   switch (timerType) {
     case CONST_INTERVAL_TIMER:
       process_interval_timer(id, timer);
@@ -75,12 +83,23 @@ void setIntervalTimer(unsigned long id) {
   setIntervalTimer(id, millis());
 }
 
+void setIntervalTimerAt(unsigned long id, unsigned long newtimer) {
+  setNewTimerAt(getMixedId(CONST_INTERVAL_TIMER, id), newtimer);
+}
+
 void setIntervalTimerOverride(unsigned long id, unsigned long msecFromNow) {
   unsigned long timer = millis();
   setNextTimeInterval(timer, msecFromNow);
   setNewTimerAt(getMixedId(CONST_INTERVAL_TIMER, id), timer);
 }
 
+void scheduleNextDelayQueue(unsigned long id, unsigned long nextTime) {
+  if (nextTime != 0) {
+    // Schedule for next process run.
+    setIntervalTimerAt(id, nextTime);
+  }
+}
+
 void setIntervalTimer(unsigned long id, unsigned long lasttimer) {
   // Set the initial timers for the regular runs
   unsigned long interval = 0;
@@ -91,6 +110,20 @@ void setIntervalTimer(unsigned long id, unsigned long lasttimer) {
     case TIMER_30SEC:      interval = 30000; break;
     case TIMER_MQTT:       interval = timermqtt_interval; break;
     case TIMER_STATISTICS: interval = 30000; break;
+    // Fall-through for all DelayQueue, which are just the fall-back timers.
+    // The timers for all delay queues will be set according to their own settings as long as there is something to process.
+    case TIMER_MQTT_DELAY_QUEUE:
+    case TIMER_C001_DELAY_QUEUE:
+    case TIMER_C003_DELAY_QUEUE:
+    case TIMER_C004_DELAY_QUEUE:
+    case TIMER_C007_DELAY_QUEUE:
+    case TIMER_C008_DELAY_QUEUE:
+    case TIMER_C009_DELAY_QUEUE:
+    case TIMER_C010_DELAY_QUEUE:
+    case TIMER_C011_DELAY_QUEUE:
+    case TIMER_C012_DELAY_QUEUE:
+    case TIMER_C013_DELAY_QUEUE:
+      interval = 1000; break;
   }
   unsigned long timer = lasttimer;
   setNextTimeInterval(timer, interval);
@@ -98,6 +131,8 @@ void setIntervalTimer(unsigned long id, unsigned long lasttimer) {
 }
 
 void process_interval_timer(unsigned long id, unsigned long lasttimer) {
+  // Set the interval timer now, it may be altered by the commands below.
+  // This is the default next-run-time.
   setIntervalTimer(id, lasttimer);
   switch (id) {
     case TIMER_20MSEC:
@@ -119,6 +154,63 @@ void process_interval_timer(unsigned long id, unsigned long lasttimer) {
     case TIMER_STATISTICS:
       logTimerStatistics();
       break;
+    case TIMER_MQTT_DELAY_QUEUE:
+      processMQTTdelayQueue();
+      break;
+  #ifdef USES_C001
+    case TIMER_C001_DELAY_QUEUE:
+      process_c001_delay_queue();
+      break;
+  #endif
+  #ifdef USES_C003
+    case TIMER_C003_DELAY_QUEUE:
+      process_c003_delay_queue();
+      break;
+  #endif
+  #ifdef USES_C004
+    case TIMER_C004_DELAY_QUEUE:
+      process_c004_delay_queue();
+      break;
+  #endif
+  #ifdef USES_C007
+    case TIMER_C007_DELAY_QUEUE:
+      process_c007_delay_queue();
+      break;
+  #endif
+  #ifdef USES_C008
+    case TIMER_C008_DELAY_QUEUE:
+      process_c008_delay_queue();
+      break;
+  #endif
+  #ifdef USES_C009
+    case TIMER_C009_DELAY_QUEUE:
+      process_c009_delay_queue();
+      break;
+  #endif
+  #ifdef USES_C010
+    case TIMER_C010_DELAY_QUEUE:
+      process_c010_delay_queue();
+      break;
+  #endif
+  #ifdef USES_C011
+    case TIMER_C011_DELAY_QUEUE:
+      process_c011_delay_queue();
+      break;
+  #endif
+  #ifdef USES_C012
+    case TIMER_C012_DELAY_QUEUE:
+      process_c012_delay_queue();
+      break;
+  #endif
+/*
+  #ifdef USES_C013
+    case TIMER_C013_DELAY_QUEUE:
+      process_c013_delay_queue();
+      break;
+  #endif
+*/
+// When extending this, also extend in _CPlugin_Helper.h
+// Look for DEFINE_Cxxx_DELAY_QUEUE_MACRO
   }
 }
 
@@ -213,18 +305,37 @@ void schedule_all_task_device_timers() {
   }
 }
 
-void schedule_task_device_timer(unsigned long task_index, unsigned long runAt) {
-/*
-  String log = F("schedule_task_device_timer: task: ");
-  log += task_index;
-  log += F(" @ ");
-  log += runAt;
-  if (Settings.TaskDeviceEnabled[task_index]) {
-    log += F(" (enabled)");
+void schedule_all_tasks_using_MQTT_controller() {
+  int ControllerIndex = firstEnabledMQTTController();
+  if (ControllerIndex < 0) return;
+  for (byte task = 0; task < TASKS_MAX; task++) {
+    if (Settings.TaskDeviceSendData[ControllerIndex][task] &&
+        Settings.ControllerEnabled[ControllerIndex] &&
+        Settings.Protocol[ControllerIndex])
+    {
+      schedule_task_device_timer_at_init(task);
+    }
   }
-  addLog(LOG_LEVEL_INFO, log);
-*/
+}
+
+void schedule_task_device_timer(unsigned long task_index, unsigned long runAt) {
+  /*
+    String log = F("schedule_task_device_timer: task: ");
+    log += task_index;
+    log += F(" @ ");
+    log += runAt;
+    if (Settings.TaskDeviceEnabled[task_index]) {
+      log += F(" (enabled)");
+    }
+    addLog(LOG_LEVEL_INFO, log);
+  */
+
   if (task_index >= TASKS_MAX) return;
+  byte DeviceIndex = getDeviceIndex(Settings.TaskDeviceNumber[task_index]);
+  if (!Device[DeviceIndex].TimerOption) return;
+  if (Device[DeviceIndex].TimerOptional && Settings.TaskDeviceTimer[task_index] == 0) {
+    return;
+  }
   if (Settings.TaskDeviceEnabled[task_index]) {
     setNewTimerAt(getMixedId(TASK_DEVICE_TIMER, task_index), runAt);
   }

src/StringConverter.ino

@@ -131,7 +131,7 @@ String toString(float value, byte decimals)
 
 String toString(WiFiMode_t mode)
 {
-  String result = F("Undefinited");
+  String result = F("Undefined");
   switch (mode)
   {
     case WIFI_OFF:
@@ -165,11 +165,13 @@ String doFormatUserVar(byte TaskIndex, byte rel_index, bool mustCheck, bool& isv
   const byte DeviceIndex = getDeviceIndex(Settings.TaskDeviceNumber[TaskIndex]);
   if (Device[DeviceIndex].ValueCount <= rel_index) {
     isvalid = false;
-    String log = F("No sensor value for TaskIndex: ");
-    log += TaskIndex;
-    log += F(" varnumber: ");
-    log += rel_index;
-    addLog(LOG_LEVEL_ERROR, log);
+    if (loglevelActiveFor(LOG_LEVEL_ERROR)) {
+      String log = F("No sensor value for TaskIndex: ");
+      log += TaskIndex;
+      log += F(" varnumber: ");
+      log += rel_index;
+      addLog(LOG_LEVEL_ERROR, log);
+    }
     return "";
   }
   if (Device[DeviceIndex].VType == SENSOR_TYPE_LONG) {
@@ -178,11 +180,13 @@ String doFormatUserVar(byte TaskIndex, byte rel_index, bool mustCheck, bool& isv
   float f(UserVar[BaseVarIndex + rel_index]);
   if (mustCheck && !isValidFloat(f)) {
     isvalid = false;
-    String log = F("Invalid float value for TaskIndex: ");
-    log += TaskIndex;
-    log += F(" varnumber: ");
-    log += rel_index;
-    addLog(LOG_LEVEL_DEBUG, log);
+    if (loglevelActiveFor(LOG_LEVEL_DEBUG)) {
+      String log = F("Invalid float value for TaskIndex: ");
+      log += TaskIndex;
+      log += F(" varnumber: ");
+      log += rel_index;
+      addLog(LOG_LEVEL_DEBUG, log);
+    }
     f = 0;
   }
   return toString(f, ExtraTaskSettings.TaskDeviceValueDecimals[rel_index]);
@@ -210,13 +214,11 @@ String formatUserVar(struct EventStruct *event, byte rel_index, bool& isvalid)
 /*********************************************************************************************\
    Wrap a string with given pre- and postfix string.
   \*********************************************************************************************/
-String wrap_String(const String& string, const String& wrap) {
-  String result;
-  result.reserve(string.length() + 2* wrap.length());
-  result = wrap;
+
+void wrap_String(const String& string, const String& wrap, String& result) {
+  result += wrap;
   result += string;
   result += wrap;
-  return result;
 }
 
 /*********************************************************************************************\
@@ -225,19 +227,25 @@ String wrap_String(const String& string, const String& wrap) {
 String to_json_object_value(const String& object, const String& value) {
   String result;
   result.reserve(object.length() + value.length() + 6);
-  result = wrap_String(object, F("\""));
+  wrap_String(object, F("\""), result);
   result += F(":");
-  if (value.length() == 0 || !isFloat(value)) {
-    if (value.indexOf('\n') == -1 && value.indexOf('"') == -1 && value.indexOf(F("Pragma")) == -1) {
-      result += wrap_String(value, F("\""));
-    } else {
+  if (value.length() == 0) {
+    // Empty string
+    result += F("\"\"");
+  } else if (!isFloat(value)) {
+    // Is not a numerical value, thus wrap with quotes
+    if (value.indexOf('\n') != -1 || value.indexOf('\r') != -1 || value.indexOf('"') != -1) {
+      // Must replace characters, so make a deepcopy
       String tmpValue(value);
       tmpValue.replace('\n', '^');
+      tmpValue.replace('\r', '^');
       tmpValue.replace('"', '\'');
-      tmpValue.replace(F("Pragma"), F("Bugje!"));
-      result += wrap_String(tmpValue, F("\""));
+      wrap_String(tmpValue, F("\""), result);
+    } else {
+      wrap_String(value, F("\""), result);
     }
   } else {
+    // It is a numerical
     result += value;
   }
   return result;
@@ -373,8 +381,32 @@ void htmlEscape(String & html)
   html.replace("'",  F("&#039;"));
   html.replace("<",  F("&lt;"));
   html.replace(">",  F("&gt;"));
+  html.replace("/", F("&#047;"));
 }
 
+void htmlStrongEscape(String & html)
+{
+  String escaped;
+  escaped.reserve(html.length());
+  for (unsigned i = 0; i < html.length(); ++i)
+  {
+    if ((html[i] >= 'a' && html[i] <= 'z') || (html[i] >= 'A' && html[i] <= 'Z') || (html[i] >= '0' && html[i] <= '9'))
+    {
+      escaped += html[i];
+    }
+    else
+    {
+      char s [4];
+      sprintf(s, "%03d", static_cast<int>(html[i]));
+      escaped += "&#";
+      escaped += s;
+      escaped += ";";
+    }
+  }
+  html = escaped;
+}
+
+
 /********************************************************************************************\
   replace other system variables like %sysname%, %systime%, %ip%
   \*********************************************************************************************/
@@ -546,11 +578,13 @@ String getReplacementString(const String& format, String& s) {
   int startpos = s.indexOf(format);
   int endpos = s.indexOf('%', startpos + 1);
   String R = s.substring(startpos, endpos + 1);
-  String log = F("ReplacementString SunTime: ");
-  log += R;
-  log += F(" offset: ");
-  log += getSecOffset(R);
-  addLog(LOG_LEVEL_DEBUG, log);
+  if (loglevelActiveFor(LOG_LEVEL_DEBUG)) {
+    String log = F("ReplacementString SunTime: ");
+    log += R;
+    log += F(" offset: ");
+    log += getSecOffset(R);
+    addLog(LOG_LEVEL_DEBUG, log);
+  }
   return R;
 }
 

src/TimeESPeasy.ino

@@ -61,8 +61,8 @@ void calcSunRiseAndSet() {
 	float eqt = equationOfTime(doy);
 	float dec = sunDeclination(doy);
 	float da = diurnalArc(dec, Settings.Latitude);
-	float rise = 12 - da - eqt - Settings.Longitude / 15.0;
-	float set = 12 + da - eqt - Settings.Longitude / 15.0;
+	float rise = 12 - da - eqt;
+	float set = 12 + da - eqt;
   tsRise.Hour = (int)rise;
   tsRise.Minute = (rise - (int)rise) * 60.0;
   tsSet.Hour = (int)set;
@@ -70,6 +70,10 @@ void calcSunRiseAndSet() {
   tsRise.Day = tsSet.Day = tm.Day;
   tsRise.Month = tsSet.Month = tm.Month;
   tsRise.Year = tsSet.Year = tm.Year;
+  // Now apply the longitude
+	int secOffset_longitude = -1.0 * (Settings.Longitude / 15.0) * 3600;
+	tsSet = addSeconds(tsSet, secOffset_longitude, false);
+	tsRise = addSeconds(tsRise, secOffset_longitude, false);
   breakTime(toLocal(makeTime(tsRise)), sunRise);
   breakTime(toLocal(makeTime(tsSet)), sunSet);
 }
@@ -143,20 +147,6 @@ void breakTime(unsigned long timeInput, struct timeStruct &tm) {
   tm.Day = time + 1;     // day of month
 }
 
-void setTime(unsigned long t) {
-  sysTime = (uint32_t)t;
-  applyTimeZone(t);
-  nextSyncTime = (uint32_t)t + syncInterval;
-  prevMillis = millis();  // restart counting from now (thanks to Korman for this fix)
-  if (Settings.UseRules)
-  {
-    static bool firstUpdate = true;
-    String event = firstUpdate ? F("Time#Initialized") : F("Time#Set");
-    firstUpdate = false;
-    rulesProcessing(event);
-  }
-}
-
 uint32_t getUnixTime() {
   return sysTime;
 }
@@ -218,13 +208,23 @@ unsigned long now() {
     unsigned long  t = getNtpTime();
     if (t != 0) {
       timeSynced = true;
-      setTime(t);
+			sysTime = (uint32_t)t;
+			applyTimeZone(t);
+			nextSyncTime = (uint32_t)t + syncInterval;
+			prevMillis = millis();  // restart counting from now (thanks to Korman for this fix)
     }
   }
   uint32_t localSystime = toLocal(sysTime);
   breakTime(localSystime, tm);
   if (timeSynced) {
     calcSunRiseAndSet();
+		if (Settings.UseRules)
+		{
+			static bool firstUpdate = true;
+			String event = firstUpdate ? F("Time#Initialized") : F("Time#Set");
+			firstUpdate = false;
+			rulesProcessing(event);
+		}
   }
   return (unsigned long)localSystime;
 }

src/WebServer.ino

@@ -875,6 +875,13 @@ void handle_root() {
     TXBuffer += F(" - ");
     TXBuffer += String(lowestRAMfunction);
     TXBuffer += F(")");
+    html_TR_TD(); TXBuffer += F("Free Stack:<TD>");
+    TXBuffer += String(getCurrentFreeStack());
+    TXBuffer += F(" (");
+    TXBuffer += String(lowestFreeStack);
+    TXBuffer += F(" - ");
+    TXBuffer += String(lowestFreeStackfunction);
+    TXBuffer += F(")");
 
     html_TR_TD(); TXBuffer += F("IP:<TD>");
     TXBuffer += formatIP(ip);
@@ -883,7 +890,9 @@ void handle_root() {
     if (wifiStatus == ESPEASY_WIFI_SERVICES_INITIALIZED)
     {
       TXBuffer += String(WiFi.RSSI());
-      TXBuffer += F(" dB");
+      TXBuffer += F(" dB (");
+      TXBuffer += WiFi.SSID();
+      TXBuffer += ')';
     }
 
     #ifdef FEATURE_MDNS
@@ -900,25 +909,31 @@ void handle_root() {
     addButton(F("sysinfo"), F("More info"));
 
     TXBuffer += F("</table><BR><BR><table class='multirow'><TR><TH>Node List:<TH>Name<TH>Build<TH>Type<TH>IP<TH>Age");
-    for (byte x = 0; x < UNIT_MAX; x++)
+    for (NodesMap::iterator it = Nodes.begin(); it != Nodes.end(); ++it)
     {
-      if (Nodes[x].ip[0] != 0)
+      if (it->second.ip[0] != 0)
       {
         char url[80];
-        sprintf_P(url, PSTR("<a class='button link' href='http://%u.%u.%u.%u'>%u.%u.%u.%u</a>"), Nodes[x].ip[0], Nodes[x].ip[1], Nodes[x].ip[2], Nodes[x].ip[3], Nodes[x].ip[0], Nodes[x].ip[1], Nodes[x].ip[2], Nodes[x].ip[3]);
-        html_TR_TD(); TXBuffer += F("Unit ");
-        TXBuffer += String(x);
-        html_TD();
-        if (x != Settings.Unit)
-          TXBuffer += Nodes[x].nodeName;
+        sprintf_P(url, PSTR("<a class='button link' href='http://%u.%u.%u.%u'>%u.%u.%u.%u</a>"), it->second.ip[0], it->second.ip[1], it->second.ip[2], it->second.ip[3], it->second.ip[0], it->second.ip[1], it->second.ip[2], it->second.ip[3]);
+        bool isThisUnit = it->first == Settings.Unit;
+        if (isThisUnit)
+          html_TR_TD_highlight();
         else
+          html_TR_TD();
+
+        TXBuffer += F("Unit ");
+        TXBuffer += String(it->first);
+        html_TD();
+        if (isThisUnit)
           TXBuffer += Settings.Name;
+        else
+          TXBuffer += it->second.nodeName;
         html_TD();
-        if (Nodes[x].build)
-          TXBuffer += String(Nodes[x].build);
+        if (it->second.build)
+          TXBuffer += String(it->second.build);
         html_TD();
-        if (Nodes[x].nodeType)
-          switch (Nodes[x].nodeType)
+        if (it->second.nodeType)
+          switch (it->second.nodeType)
           {
             case NODE_TYPE_ID_ESP_EASY_STD:
               TXBuffer += F("ESP Easy");
@@ -939,7 +954,7 @@ void handle_root() {
         html_TD();
         TXBuffer += url;
         html_TD();
-        TXBuffer += String( Nodes[x].age);
+        TXBuffer += String( it->second.age);
       }
     }
 
@@ -1027,6 +1042,7 @@ void handle_config() {
       MQTTclient_should_reconnect = true;
     }
     strncpy(Settings.Name, name.c_str(), sizeof(Settings.Name));
+    Settings.appendUnitToHostname(isFormItemChecked(F("appendunittohostname")));
     //strncpy(SecuritySettings.Password, password.c_str(), sizeof(SecuritySettings.Password));
     copyFormPassword(F("password"), SecuritySettings.Password, sizeof(SecuritySettings.Password));
     strncpy(SecuritySettings.WifiSSID, ssid.c_str(), sizeof(SecuritySettings.WifiSSID));
@@ -1076,7 +1092,8 @@ void handle_config() {
   Settings.Name[25] = 0;
   SecuritySettings.Password[25] = 0;
   addFormTextBox( F("Unit Name"), F("name"), Settings.Name, 25);
-  addFormNumericBox( F("Unit Number"), F("unit"), Settings.Unit, 0, 9999);
+  addFormNumericBox( F("Unit Number"), F("unit"), Settings.Unit, 0, UNIT_NUMBER_MAX);
+  addFormCheckBox(F("Append Unit Number to hostname"), F("appendunittohostname"), Settings.appendUnitToHostname());
   addFormPasswordBox(F("Admin Password"), F("password"), SecuritySettings.Password, 25);
 
   addFormSubHeader(F("Wifi Settings"));
@@ -1164,6 +1181,12 @@ void handle_controllers() {
   String MQTTLwtTopic = WebServer.arg(F("mqttlwttopic"));
   String lwtmessageconnect = WebServer.arg(F("lwtmessageconnect"));
   String lwtmessagedisconnect = WebServer.arg(F("lwtmessagedisconnect"));
+  const int minimumsendinterval = getFormItemInt(F("minimumsendinterval"), 100);
+  const int maxqueuedepth = getFormItemInt(F("maxqueuedepth"), 10);
+  const int maxretry = getFormItemInt(F("maxretry"), 10);
+  String deleteoldest = WebServer.arg(F("deleteoldest"));
+  String mustcheckreply = WebServer.arg(F("mustcheckreply"));
+  const int clienttimeout = getFormItemInt(F("clienttimeout"), CONTROLLER_CLIENTTIMEOUT_DFLT);
 
 
   //submitted data
@@ -1182,6 +1205,9 @@ void handle_controllers() {
         //reset (some) default-settings
         byte ProtocolIndex = getProtocolIndex(Settings.Protocol[controllerindex]);
         ControllerSettings.Port = Protocol[ProtocolIndex].defaultPort;
+        ControllerSettings.MinimalTimeBetweenMessages = CONTROLLER_DELAY_QUEUE_DELAY_DFLT;
+        ControllerSettings.ClientTimeout = CONTROLLER_CLIENTTIMEOUT_DFLT;
+//        ControllerSettings.MaxQueueDepth = 0;
         if (Protocol[ProtocolIndex].usesTemplate)
           CPlugin_ptr[ProtocolIndex](CPLUGIN_PROTOCOL_TEMPLATE, &TempEvent, dummyString);
         strncpy(ControllerSettings.Subscribe, TempEvent.String1.c_str(), sizeof(ControllerSettings.Subscribe));
@@ -1241,11 +1267,18 @@ void handle_controllers() {
         strncpy(ControllerSettings.MQTTLwtTopic, MQTTLwtTopic.c_str(), sizeof(ControllerSettings.MQTTLwtTopic));
         strncpy(ControllerSettings.LWTMessageConnect, lwtmessageconnect.c_str(), sizeof(ControllerSettings.LWTMessageConnect));
         strncpy(ControllerSettings.LWTMessageDisconnect, lwtmessagedisconnect.c_str(), sizeof(ControllerSettings.LWTMessageDisconnect));
+        ControllerSettings.MinimalTimeBetweenMessages = minimumsendinterval;
+        ControllerSettings.MaxQueueDepth = maxqueuedepth;
+        ControllerSettings.MaxRetry = maxretry;
+        ControllerSettings.DeleteOldest = deleteoldest.toInt();
+        ControllerSettings.MustCheckReply = mustcheckreply.toInt();
+        ControllerSettings.ClientTimeout = clienttimeout;
+
 
         CPlugin_ptr[ProtocolIndex](CPLUGIN_INIT, &TempEvent, dummyString);
       }
     }
-    addHtmlError(SaveControllerSettings(controllerindex, (byte*)&ControllerSettings, sizeof(ControllerSettings)));
+    addHtmlError(SaveControllerSettings(controllerindex, ControllerSettings));
     addHtmlError(SaveSettings());
   }
 
@@ -1259,7 +1292,7 @@ void handle_controllers() {
     ControllerSettingsStruct ControllerSettings;
     for (byte x = 0; x < CONTROLLER_MAX; x++)
     {
-      LoadControllerSettings(x, (byte*)&ControllerSettings, sizeof(ControllerSettings));
+      LoadControllerSettings(x, ControllerSettings);
       html_TR_TD();
       TXBuffer += F("<a class='button link' href=\"controllers?index=");
       TXBuffer += x + 1;
@@ -1315,18 +1348,28 @@ void handle_controllers() {
     if (Settings.Protocol[controllerindex])
     {
       ControllerSettingsStruct ControllerSettings;
-      LoadControllerSettings(controllerindex, (byte*)&ControllerSettings, sizeof(ControllerSettings));
+      LoadControllerSettings(controllerindex, ControllerSettings);
       byte choice = ControllerSettings.UseDNS;
       String options[2];
       options[0] = F("Use IP address");
       options[1] = F("Use Hostname");
 
+      byte choice_delete_oldest = ControllerSettings.DeleteOldest;
+      String options_delete_oldest[2];
+      options_delete_oldest[0] = F("Ignore New");
+      options_delete_oldest[1] = F("Delete Oldest");
+
+      byte choice_mustcheckreply = ControllerSettings.MustCheckReply;
+      String options_mustcheckreply[2];
+      options_mustcheckreply[0] = F("Ignore Acknowledgement");
+      options_mustcheckreply[1] = F("Check Acknowledgement");
+
+
       byte ProtocolIndex = getProtocolIndex(Settings.Protocol[controllerindex]);
       if (!Protocol[ProtocolIndex].Custom)
       {
 
         addFormSelector(F("Locate Controller"), F("usedns"), 2, options, NULL, NULL, choice, true);
-
         if (ControllerSettings.UseDNS)
         {
           addFormTextBox( F("Controller Hostname"), F("controllerhostname"), ControllerSettings.HostName, sizeof(ControllerSettings.HostName)-1);
@@ -1337,15 +1380,24 @@ void handle_controllers() {
         }
 
         addFormNumericBox( F("Controller Port"), F("controllerport"), ControllerSettings.Port, 1, 65535);
+        addFormNumericBox( F("Minimum Send Interval"), F("minimumsendinterval"), ControllerSettings.MinimalTimeBetweenMessages, 1, CONTROLLER_DELAY_QUEUE_DELAY_MAX);
+        addUnit(F("ms"));
+        addFormNumericBox( F("Max Queue Depth"), F("maxqueuedepth"), ControllerSettings.MaxQueueDepth, 1, CONTROLLER_DELAY_QUEUE_DEPTH_MAX);
+        addFormNumericBox( F("Max Retries"), F("maxretry"), ControllerSettings.MaxRetry, 1, CONTROLLER_DELAY_QUEUE_RETRY_MAX);
+        addFormSelector(F("Full Queue Action"), F("deleteoldest"), 2, options_delete_oldest, NULL, NULL, choice_delete_oldest, true);
+
+        addFormSelector(F("Check Reply"), F("mustcheckreply"), 2, options_mustcheckreply, NULL, NULL, choice_mustcheckreply, true);
+        addFormNumericBox( F("Client Timeout"), F("clienttimeout"), ControllerSettings.ClientTimeout, 10, CONTROLLER_CLIENTTIMEOUT_MAX);
+        addUnit(F("ms"));
 
         if (Protocol[ProtocolIndex].usesAccount)
         {
-           String protoDisplayName;
+          String protoDisplayName;
           if (!getControllerProtocolDisplayName(ProtocolIndex, CONTROLLER_USER, protoDisplayName)) {
             protoDisplayName = F("Controller User");
           }
           addFormTextBox(protoDisplayName, F("controlleruser"), SecuritySettings.ControllerUser[controllerindex], sizeof(SecuritySettings.ControllerUser[0])-1);
-         }
+        }
         if (Protocol[ProtocolIndex].usesPassword)
         {
           String protoDisplayName;
@@ -1428,6 +1480,11 @@ void handle_controllers() {
 // Flash strings are not checked for duplication.
 //********************************************************************************
 
+void html_TR_TD_highlight() {
+  TXBuffer += F("<TR class=\"highlight\">");
+  html_TD();
+}
+
 void html_TR_TD() {
   TXBuffer += F("<TR>");
   html_TD();
@@ -1882,15 +1939,17 @@ void handle_devices() {
       PluginCall(PLUGIN_EXIT, &TempEvent, dummyString);
 
       taskClear(taskIndex, false); // clear settings, but do not save
+      ClearCustomTaskSettings(taskIndex);
 
       Settings.TaskDeviceNumber[taskIndex] = taskdevicenumber;
       if (taskdevicenumber != 0) // set default values if a new device has been selected
       {
         //NOTE: do not enable task by default. allow user to enter sensible valus first and let him enable it when ready.
-        if (ExtraTaskSettings.TaskDeviceValueNames[0][0] == 0) // if field set empty, reload defaults
+        if (ExtraTaskSettings.TaskIndex != taskIndex) // if field set empty, reload defaults
           PluginCall(PLUGIN_GET_DEVICEVALUENAMES, &TempEvent, dummyString); //the plugin should populate ExtraTaskSettings with its default values.
-
-        ClearCustomTaskSettings(taskIndex);
+      } else {
+        SaveTaskSettings(taskIndex);
+        SaveSettings();
       }
     }
     else if (taskdevicenumber != 0) //save settings
@@ -1951,7 +2010,7 @@ void handle_devices() {
       // }
 
       TempEvent.TaskIndex = taskIndex;
-      if (ExtraTaskSettings.TaskDeviceValueNames[0][0] == 0) // if field set empty, reload defaults
+      if (ExtraTaskSettings.TaskIndex != TempEvent.TaskIndex) // if field set empty, reload defaults
         PluginCall(PLUGIN_GET_DEVICEVALUENAMES, &TempEvent, dummyString);
 
       //allow the plugin to save plugin-specific form settings.
@@ -2248,7 +2307,7 @@ void handle_devices() {
               html_TR_TD(); TXBuffer += F("IDX:<TD>");
               id = F("TDID");   //="taskdeviceid"
               id += controllerNr + 1;
-              addNumericBox(id, Settings.TaskDeviceID[controllerNr][taskIndex], 0, 999999999); // Looks like it is an unsigned int, so could be up to 4 bln.
+              addNumericBox(id, Settings.TaskDeviceID[controllerNr][taskIndex], 0, DOMOTICZ_MAX_IDX);
             }
           }
         }
@@ -2563,8 +2622,8 @@ void addPinSelect(boolean forI2C, String name,  int choice)
 //********************************************************************************
 void addPinSelect(boolean forI2C, String name,  int choice)
 {
-  String options[PIN_D_MAX+1];
-  int optionValues[PIN_D_MAX+1];
+  String * options = new String[PIN_D_MAX+1];
+  int * optionValues = new int[PIN_D_MAX+1];
   options[0] = F("- None -");
   optionValues[0] = -1;
   for(byte x=1; x < PIN_D_MAX+1; x++)
@@ -2574,6 +2633,8 @@ void addPinSelect(boolean forI2C, String name,  int choice)
     optionValues[x] = x;
   }
   renderHTMLForPinSelect(options, optionValues, forI2C, name, choice, PIN_D_MAX+1);
+  delete[] optionValues;
+  delete[] options;
 }
 #endif
 
@@ -3084,6 +3145,7 @@ void handle_log() {
 // Web Interface JSON log page
 //********************************************************************************
 void handle_log_JSON() {
+  if (!isLoggedIn()) return;
   TXBuffer.startJsonStream();
   String webrequest = WebServer.arg(F("view"));
   TXBuffer += F("{\"Log\": {");
@@ -3094,7 +3156,7 @@ void handle_log_JSON() {
         TXBuffer += ',';
       TXBuffer += '{';
       int loglevel;
-      stream_next_json_object_value(F("label"), getLogLevelDisplayString(i, loglevel));
+      stream_next_json_object_value(F("label"), getLogLevelDisplayStringFromIndex(i, loglevel));
       stream_last_json_object_value(F("loglevel"), String(loglevel));
     }
     TXBuffer += F("],\n");
@@ -3137,6 +3199,7 @@ void handle_log_JSON() {
   stream_last_json_object_value(F("logTimeSpan"), String(logTimeSpan));
   TXBuffer += F("}\n");
   TXBuffer.endStream();
+  updateLogLevelCache();
 }
 
 //********************************************************************************
@@ -3613,6 +3676,7 @@ void handle_control() {
   }
   else if (command.equalsIgnoreCase(F("taskrun")) ||
            command.equalsIgnoreCase(F("taskvalueset")) ||
+           command.equalsIgnoreCase(F("taskvaluetoggle")) ||
            command.equalsIgnoreCase(F("rules"))) {
     addLog(LOG_LEVEL_INFO,String(F("HTTP : ")) + webrequest);
     ExecuteCommand(VALUE_SOURCE_HTTP,webrequest.c_str());
@@ -3754,14 +3818,14 @@ void handle_json()
     }
     if(showNodes) {
       bool comma_between=false;
-      for (byte x = 0; x < UNIT_MAX; x++)
+      for (NodesMap::iterator it = Nodes.begin(); it != Nodes.end(); ++it)
       {
-        if (Nodes[x].ip[0] != 0)
+        if (it->second.ip[0] != 0)
         {
 
           char ip[20];
 
-          sprintf_P(ip, PSTR("%u.%u.%u.%u"), Nodes[x].ip[0], Nodes[x].ip[1], Nodes[x].ip[2], Nodes[x].ip[3]);
+          sprintf_P(ip, PSTR("%u.%u.%u.%u"), it->second.ip[0], it->second.ip[1], it->second.ip[2], it->second.ip[3]);
 
           if( comma_between ) {
             TXBuffer += F(",");
@@ -3771,17 +3835,17 @@ void handle_json()
           }
 
           TXBuffer += F("{");
-          stream_next_json_object_value(F("nr"), String(x));
+          stream_next_json_object_value(F("nr"), String(it->first));
           stream_next_json_object_value(F("name"),
-              (x != Settings.Unit) ? Nodes[x].nodeName : Settings.Name);
+              (it->first != Settings.Unit) ? it->second.nodeName : Settings.Name);
 
-          if (Nodes[x].build) {
-            stream_next_json_object_value(F("build"), String(Nodes[x].build));
+          if (it->second.build) {
+            stream_next_json_object_value(F("build"), String(it->second.build));
           }
 
-          if (Nodes[x].nodeType) {
+          if (it->second.nodeType) {
             String platform;
-            switch (Nodes[x].nodeType)
+            switch (it->second.nodeType)
             {
               case NODE_TYPE_ID_ESP_EASY_STD:     platform = F("ESP Easy");      break;
               case NODE_TYPE_ID_ESP_EASYM_STD:    platform = F("ESP Easy Mega"); break;
@@ -3793,7 +3857,7 @@ void handle_json()
               stream_next_json_object_value(F("platform"), platform);
           }
           stream_next_json_object_value(F("ip"), ip);
-          stream_last_json_object_value(F("age"),  String( Nodes[x].age ));
+          stream_last_json_object_value(F("age"),  String( it->second.age ));
         } // if node info exists
       } // for loop
       if(comma_between) {
@@ -4079,10 +4143,10 @@ void addLogLevelSelect(String name, int choice)
 {
   String options[LOG_LEVEL_NRELEMENTS + 1];
   int optionValues[LOG_LEVEL_NRELEMENTS + 1] = {0};
-  options[0] = F("None");
+  options[0] = getLogLevelDisplayString(0);
   optionValues[0] = 0;
   for (int i = 0; i < LOG_LEVEL_NRELEMENTS; ++i) {
-    options[i + 1] = getLogLevelDisplayString(i, optionValues[i + 1]);
+    options[i + 1] = getLogLevelDisplayStringFromIndex(i, optionValues[i + 1]);
   }
   addSelector(name, LOG_LEVEL_NRELEMENTS + 1, options, optionValues, NULL, choice, false);
 }
@@ -4154,6 +4218,7 @@ void handle_download()
 
   WebServer.sendHeader(F("Content-Disposition"), str);
   WebServer.streamFile(dataFile, F("application/octet-stream"));
+  dataFile.close();
 }
 
 
@@ -4379,16 +4444,19 @@ boolean handle_custom(String path) {
     if(!unit) unit = btnunit; // unit element prevails, if not used then set to btnunit
     if (unit && unit != Settings.Unit)
     {
-      TXBuffer.startStream();
-      sendHeadandTail(F("TmplDsh"),_HEAD);
-      char url[40];
-      sprintf_P(url, PSTR("http://%u.%u.%u.%u/dashboard.esp"), Nodes[unit].ip[0], Nodes[unit].ip[1], Nodes[unit].ip[2], Nodes[unit].ip[3]);
-      TXBuffer += F("<meta http-equiv=\"refresh\" content=\"0; URL=");
-      TXBuffer += url;
-      TXBuffer += F("\">");
-      sendHeadandTail(F("TmplDsh"),_TAIL);
-      TXBuffer.endStream();
-      return true;
+      NodesMap::iterator it = Nodes.find(unit);
+      if (it != Nodes.end()) {
+        TXBuffer.startStream();
+        sendHeadandTail(F("TmplDsh"),_HEAD);
+        char url[40];
+        sprintf_P(url, PSTR("http://%u.%u.%u.%u/dashboard.esp"), it->second.ip[0], it->second.ip[1], it->second.ip[2], it->second.ip[3]);
+        TXBuffer += F("<meta http-equiv=\"refresh\" content=\"0; URL=");
+        TXBuffer += url;
+        TXBuffer += F("\">");
+        sendHeadandTail(F("TmplDsh"),_TAIL);
+        TXBuffer.endStream();
+        return true;
+      }
     }
 
     TXBuffer.startStream();
@@ -4402,16 +4470,16 @@ boolean handle_custom(String path) {
     // create unit selector dropdown
     addSelector_Head(F("unit"), true);
     byte choice = Settings.Unit;
-    for (byte x = 0; x < UNIT_MAX; x++)
+    for (NodesMap::iterator it = Nodes.begin(); it != Nodes.end(); ++it)
     {
-      if (Nodes[x].ip[0] != 0 || x == Settings.Unit)
+      if (it->second.ip[0] != 0 || it->first == Settings.Unit)
       {
-        String name = String(x) + F(" - ");
-        if (x != Settings.Unit)
-          name += Nodes[x].nodeName;
+        String name = String(it->first) + F(" - ");
+        if (it->first != Settings.Unit)
+          name += it->second.nodeName;
         else
           name += Settings.Name;
-        addSelector_Item(name, x, choice == x, false, F(""));
+        addSelector_Item(name, it->first, choice == it->first, false, F(""));
       }
     }
     addSelector_Foot();
@@ -4419,10 +4487,19 @@ boolean handle_custom(String path) {
     // create <> navigation buttons
     byte prev=Settings.Unit;
     byte next=Settings.Unit;
-    for (byte x = Settings.Unit-1; x > 0; x--)
-      if (Nodes[x].ip[0] != 0) {prev = x; break;}
-    for (byte x = Settings.Unit+1; x < UNIT_MAX; x++)
-      if (Nodes[x].ip[0] != 0) {next = x; break;}
+    NodesMap::iterator it;
+    for (byte x = Settings.Unit-1; x > 0; x--) {
+      it = Nodes.find(x);
+      if (it != Nodes.end()) {
+        if (it->second.ip[0] != 0) {prev = x; break;}
+      }
+    }
+    for (byte x = Settings.Unit+1; x < UNIT_MAX; x++) {
+      it = Nodes.find(x);
+      if (it != Nodes.end()) {
+        if (it->second.ip[0] != 0) {next = x; break;}
+      }
+    }
 
     TXBuffer += F("<a class='button link' href=");
     TXBuffer += path;
@@ -4521,18 +4598,40 @@ void handle_filelist() {
     checkRuleSets();
   }
 
+  const int pageSize = 25;
+  int startIdx = 0;
 
+  String fstart = WebServer.arg(F("start"));
+  if (fstart.length() > 0)
+  {
+    startIdx = atoi(fstart.c_str());
+  }
+  int endIdx = startIdx + pageSize - 1;
 
   TXBuffer += F("<table class='multirow' border=1px frame='box' rules='all'><TH style='width:50px;'><TH>Filename<TH style='width:80px;'>Size");
 
   fs::Dir dir = SPIFFS.openDir("");
+
+  int count = -1;
   while (dir.next())
   {
+    ++count;
+
+    if (count < startIdx)
+    {
+      continue;
+    }
+
     html_TR_TD();
     if (dir.fileName() != F(FILE_CONFIG) && dir.fileName() != F(FILE_SECURITY) && dir.fileName() != F(FILE_NOTIFICATION))
     {
       TXBuffer += F("<a class='button link' href=\"filelist?delete=");
       TXBuffer += dir.fileName();
+      if (startIdx > 0)
+      {
+        TXBuffer += F("&start=");
+        TXBuffer += startIdx;
+      }
       TXBuffer += F("\">Del</a>");
     }
 
@@ -4543,12 +4642,32 @@ void handle_filelist() {
     TXBuffer += F("</a>");
     fs::File f = dir.openFile("r");
     html_TD();
-    TXBuffer += f.size();
+    if (f) {
+      TXBuffer += f.size();
+      f.close();
+    }
+    if (count >= endIdx)
+    {
+      break;
+    }
   }
   TXBuffer += F("</table></form>");
-  TXBuffer += F("<BR><a class='button link' href=\"/upload\">Upload</a><BR><BR>");
-    sendHeadandTail(F("TmplStd"),true);
-    TXBuffer.endStream();
+  TXBuffer += F("<BR><a class='button link' href=\"/upload\">Upload</a>");
+  if (startIdx > 0)
+  {
+    TXBuffer += F("<a class='button link' href=\"/filelist?start=");
+    TXBuffer += max(0, startIdx - pageSize);
+    TXBuffer += F("\">Previous</a>");
+  }
+  if (count >= endIdx and dir.next())
+  {
+    TXBuffer += F("<a class='button link' href=\"/filelist?start=");
+    TXBuffer += endIdx + 1;
+    TXBuffer += F("\">Next</a>");
+  }
+  TXBuffer += F("<BR><BR>");
+  sendHeadandTail(F("TmplStd"),true);
+  TXBuffer.endStream();
 #endif
 #if defined(ESP32)
   String fdelete = WebServer.arg(F("delete"));
@@ -4559,35 +4678,67 @@ void handle_filelist() {
     // flashCount();
   }
 
+  const int pageSize = 25;
+  int startIdx = 0;
 
+  String fstart = WebServer.arg(F("start"));
+  if (fstart.length() > 0)
+  {
+    startIdx = atoi(fstart.c_str());
+  }
+  int endIdx = startIdx + pageSize - 1;
 
   TXBuffer += F("<table class='multirow' border=1px frame='box' rules='all'><TH><TH>Filename<TH>Size");
 
   File root = SPIFFS.open("/");
   File file = root.openNextFile();
-  while (file)
+  int count = -1;
+  while (file and count < endIdx)
   {
     if(!file.isDirectory()){
-      html_TR_TD();
-      if (strcmp(file.name(), FILE_CONFIG) != 0 && strcmp(file.name(), FILE_SECURITY) != 0 && strcmp(file.name(), FILE_NOTIFICATION) != 0)
-      {
-        TXBuffer += F("<a class='button link' href=\"filelist?delete=");
-        TXBuffer += file.name();
-        TXBuffer += F("\">Del</a>");
-      }
+      ++count;
 
-      TXBuffer += F("<TD><a href=\"");
-      TXBuffer += file.name();
-      TXBuffer += F("\">");
-      TXBuffer += file.name();
-      TXBuffer += F("</a>");
-      html_TD();
-      TXBuffer += file.size();
-      file = root.openNextFile();
+      if (count >= startIdx)
+      {
+        html_TR_TD();
+        if (strcmp(file.name(), FILE_CONFIG) != 0 && strcmp(file.name(), FILE_SECURITY) != 0 && strcmp(file.name(), FILE_NOTIFICATION) != 0)
+        {
+          TXBuffer += F("<a class='button link' href=\"filelist?delete=");
+          TXBuffer += file.name();
+          if (startIdx > 0)
+          {
+            TXBuffer += F("&start=");
+            TXBuffer += startIdx;
+          }
+          TXBuffer += F("\">Del</a>");
+        }
+
+        TXBuffer += F("<TD><a href=\"");
+        TXBuffer += file.name();
+        TXBuffer += F("\">");
+        TXBuffer += file.name();
+        TXBuffer += F("</a>");
+        html_TD();
+        TXBuffer += file.size();
+      }
     }
+    file = root.openNextFile();
   }
   TXBuffer += F("</table></form>");
-  TXBuffer += F("<BR><a class='button link' href=\"/upload\">Upload</a><BR><BR>");
+  TXBuffer += F("<BR><a class='button link' href=\"/upload\">Upload</a>");
+  if (startIdx > 0)
+  {
+    TXBuffer += F("<a class='button link' href=\"/filelist?start=");
+    TXBuffer += startIdx < pageSize ? 0 : startIdx - pageSize;
+    TXBuffer += F("\">Previous</a>");
+  }
+  if (count >= endIdx and file)
+  {
+    TXBuffer += F("<a class='button link' href=\"/filelist?start=");
+    TXBuffer += endIdx + 1;
+    TXBuffer += F("\">Next</a>");
+  }
+  TXBuffer += F("<BR><BR>");
     sendHeadandTail(F("TmplStd"),true);
     TXBuffer.endStream();
 #endif
@@ -4852,7 +5003,11 @@ void handle_setup() {
       {
         html_TR_TD(); TXBuffer += F("<label class='container2'>");
         TXBuffer += F("<input type='radio' name='ssid' value='");
-        TXBuffer += WiFi.SSID(i);
+        {
+          String escapeBuffer = WiFi.SSID(i);
+          htmlStrongEscape(escapeBuffer);
+          TXBuffer += escapeBuffer;
+        }
         TXBuffer += F("'");
         if (WiFi.SSID(i) == ssid)
           TXBuffer += F(" checked ");
@@ -4991,7 +5146,7 @@ void handle_rules() {
         log += F(" Create new file: ");
         log += fileName;
         fs::File f = SPIFFS.open(fileName, "w");
-        f.close();
+        if (f) f.close();
       }
     }
     addLog(LOG_LEVEL_INFO, log);
@@ -5127,6 +5282,13 @@ void handle_sysinfo() {
    TXBuffer += F(" - ");
    TXBuffer += lowestRAMfunction;
    TXBuffer += F(")");
+   html_TR_TD(); TXBuffer += F("Free Stack<TD>");
+   TXBuffer += getCurrentFreeStack();
+   TXBuffer += F(" (");
+   TXBuffer += lowestFreeStack;
+   TXBuffer += F(" - ");
+   TXBuffer += lowestFreeStackfunction;
+   TXBuffer += F(")");
 
    html_TR_TD(); TXBuffer += F("Boot<TD>");
    TXBuffer += getLastBootCauseString();
@@ -5188,16 +5350,6 @@ void handle_sysinfo() {
   html_TR_TD(); TXBuffer += F("Allowed IP Range<TD>");
   TXBuffer += describeAllowedIPrange();
 
-  html_TR_TD(); TXBuffer += F("Serial Port available:<TD>");
-  TXBuffer += String(SerialAvailableForWrite());
-  TXBuffer += F(" (");
-  #if defined(ESP8266)
-    TXBuffer += Serial.availableForWrite();
-  #endif
-  TXBuffer += F(" , ");
-  TXBuffer += Serial.available();
-  TXBuffer += F(")");
-
   html_TR_TD(); TXBuffer += F("STA MAC<TD>");
 
   uint8_t mac[] = {0, 0, 0, 0, 0, 0};
@@ -5262,6 +5414,23 @@ void handle_sysinfo() {
    TXBuffer += F("<TR><TD id='copyText_11'>Binary filename<TD id='copyText_12'>");
    TXBuffer += String(CRCValues.binaryFilename);
 
+
+   addTableSeparator(F("System Status"), 2, 3);
+   {
+     // Actual Loglevel
+     html_TR_TD(); TXBuffer += F("Syslog Log Level:<TD>");
+     TXBuffer += getLogLevelDisplayString(Settings.SyslogLevel);
+     html_TR_TD(); TXBuffer += F("Serial Log Level:<TD>");
+     TXBuffer += getLogLevelDisplayString(getSerialLogLevel());
+     html_TR_TD(); TXBuffer += F("Web Log Level:<TD>");
+     TXBuffer += getLogLevelDisplayString(getWebLogLevel());
+     #ifdef FEATURE_SD
+     html_TR_TD(); TXBuffer += F("SD Log Level:<TD>");
+     TXBuffer += getLogLevelDisplayString(Settings.SDLogLevel);
+     #endif
+   }
+
+
    addTableSeparator(F("ESP board"), 2, 3);
 
    html_TR_TD(); TXBuffer += F("ESP Chip ID<TD>");
@@ -5294,6 +5463,8 @@ void handle_sysinfo() {
      TXBuffer += ESP.getCpuFreqMHz();
      TXBuffer += F(" MHz");
   #endif
+  html_TR_TD(); TXBuffer += F("ESP Board Name:<TD>");
+  TXBuffer += ARDUINO_BOARD;
 
 
    addTableSeparator(F("Storage"), 2, 3);
@@ -5352,6 +5523,18 @@ void handle_sysinfo() {
    TXBuffer += F(" kB free)");
   #endif
 
+  html_TR_TD(); TXBuffer += F("SPIFFS Size<TD>");
+  {
+  #if defined(ESP8266)
+    fs::FSInfo fs_info;
+    SPIFFS.info(fs_info);
+    TXBuffer += fs_info.totalBytes / 1024;
+    TXBuffer += F(" kB (");
+    TXBuffer += (fs_info.totalBytes - fs_info.usedBytes) / 1024;
+    TXBuffer += F(" kB free)");
+  #endif
+  }
+
   if (showSettingsFileLayout) {
     addTableSeparator(F("Settings Files"), 2, 3);
     html_TR_TD();

src/WebStaticData.h

@@ -236,6 +236,8 @@ static const char pgDefaultCSS[] PROGMEM = {
     "table.multirow tr {padding: 4px; }"
       "table.multirow tr:nth-child(even){background-color: #DEE6FF; }"
     "table.multirow {color: #000; width: 100%; min-width: 420px; border-collapse: collapse; }"
+    // Highlight row
+    "tr.highlight td { background-color: #dbff0075; }"    
     // inside a form
     ".note {color: #444; font-style: italic; }"
     //header with title and menu

src/_C001.ino

@@ -31,49 +31,20 @@ boolean CPlugin_001(byte function, struct EventStruct *event, String& string)
         break;
       }
 
+    case CPLUGIN_INIT:
+      {
+        ControllerSettingsStruct ControllerSettings;
+        LoadControllerSettings(event->ControllerIndex, ControllerSettings);
+        C001_DelayHandler.configureControllerSettings(ControllerSettings);
+        break;
+      }
+
     case CPLUGIN_PROTOCOL_SEND:
       {
         if (event->idx != 0)
         {
-          if (!WiFiConnected(100)) {
-            success = false;
-            break;
-          }
-          ControllerSettingsStruct ControllerSettings;
-          LoadControllerSettings(event->ControllerIndex, (byte*)&ControllerSettings, sizeof (ControllerSettings));
-
-          String authHeader = "";
-          if ((SecuritySettings.ControllerUser[event->ControllerIndex][0] != 0) && (SecuritySettings.ControllerPassword[event->ControllerIndex][0] != 0))
-          {
-            base64 encoder;
-            String auth = SecuritySettings.ControllerUser[event->ControllerIndex];
-            auth += ":";
-            auth += SecuritySettings.ControllerPassword[event->ControllerIndex];
-            authHeader = F("Authorization: Basic ");
-            authHeader += encoder.encode(auth);
-            authHeader += F(" \r\n");
-          }
-
-          // boolean success = false;
-          addLog(LOG_LEVEL_DEBUG, String(F("HTTP : connecting to "))+ControllerSettings.getHostPortString());
-
-
-          // Use WiFiClient class to create TCP connections
-          WiFiClient client;
-          if (!ControllerSettings.connectToHost(client))
-          {
-            connectionFailures++;
-
-            addLog(LOG_LEVEL_ERROR, F("HTTP : connection failed"));
-            return false;
-          }
-          statusLED(true);
-          if (connectionFailures)
-            connectionFailures--;
-
           // We now create a URI for the request
-          String url = F("/json.htm?type=command&param=udevice&idx=");
-          url += event->idx;
+          String url;
 
           switch (event->sensorType)
           {
@@ -125,38 +96,8 @@ boolean CPlugin_001(byte function, struct EventStruct *event, String& string)
             url += mapVccToDomoticz();
           #endif
 
-          // This will send the request to the server
-          String request = F("GET ");
-          request += url;
-          request += F(" HTTP/1.1\r\n");
-          request += F("Host: ");
-          request += ControllerSettings.getHost();
-          request += F("\r\n");
-          request += authHeader;
-          request += F("Connection: close\r\n\r\n");
-          client.print(request);
-
-          unsigned long timer = millis() + 200;
-          while (!client.available() && !timeOutReached(timer))
-            yield();
-
-          // Read all the lines of the reply from server and log them
-          while (client.available()) {
-            // String line = client.readStringUntil('\n');
-            String line;
-            safeReadStringUntil(client, line, '\n');
-            addLog(LOG_LEVEL_DEBUG_MORE, line);
-            if (line.startsWith(F("HTTP/1.1 200 OK")) )
-            {
-              addLog(LOG_LEVEL_DEBUG, F("HTTP : Success"));
-              success = true;
-            }
-            yield();
-          }
-          addLog(LOG_LEVEL_DEBUG, F("HTTP : closing connection"));
-
-          client.flush();
-          client.stop();
+          success = C001_DelayHandler.addToQueue(C001_queue_element(event->ControllerIndex, url));
+          scheduleNextDelayQueue(TIMER_C001_DELAY_QUEUE, C001_DelayHandler.getNextScheduleTime());
         } // if ixd !=0
         else
         {
@@ -167,4 +108,17 @@ boolean CPlugin_001(byte function, struct EventStruct *event, String& string)
   }
   return success;
 }
+
+bool do_process_c001_delay_queue(int controller_number, const C001_queue_element& element, ControllerSettingsStruct& ControllerSettings) {
+  WiFiClient client;
+  if (!try_connect_host(controller_number, client, ControllerSettings))
+    return false;
+
+  // This will send the request to the server
+  String request = create_http_request_auth(controller_number, element.controller_idx, ControllerSettings, F("GET"), element.txt);
+
+  addLog(LOG_LEVEL_DEBUG, element.txt);
+  return send_via_http(controller_number, client, request, ControllerSettings.MustCheckReply);
+}
+
 #endif

src/_C002.ino

@@ -33,6 +33,14 @@ boolean CPlugin_002(byte function, struct EventStruct *event, String& string)
         break;
       }
 
+    case CPLUGIN_INIT:
+      {
+        ControllerSettingsStruct ControllerSettings;
+        LoadControllerSettings(event->ControllerIndex, ControllerSettings);
+        MQTTDelayHandler.configureControllerSettings(ControllerSettings);
+        break;
+      }
+
     case CPLUGIN_PROTOCOL_TEMPLATE:
       {
         event->String1 = F("domoticz/out");
@@ -138,11 +146,13 @@ boolean CPlugin_002(byte function, struct EventStruct *event, String& string)
         if (event->idx != 0)
         {
           ControllerSettingsStruct ControllerSettings;
-          LoadControllerSettings(event->ControllerIndex, (byte*)&ControllerSettings, sizeof(ControllerSettings));
+          LoadControllerSettings(event->ControllerIndex, ControllerSettings);
+/*
           if (!ControllerSettings.checkHostReachable(true)) {
             success = false;
             break;
           }
+*/
           StaticJsonBuffer<200> jsonBuffer;
 
           JsonObject& root = jsonBuffer.createObject();

src/_C003.ino

@@ -30,77 +30,24 @@ boolean CPlugin_003(byte function, struct EventStruct *event, String& string)
         break;
       }
 
-    case CPLUGIN_PROTOCOL_SEND:
+    case CPLUGIN_INIT:
       {
         ControllerSettingsStruct ControllerSettings;
-        LoadControllerSettings(event->ControllerIndex, (byte*)&ControllerSettings, sizeof(ControllerSettings));
-
-        boolean success = false;
-        char log[80];
-        addLog(LOG_LEVEL_DEBUG, String(F("TELNT : connecting to ")) + ControllerSettings.getHostPortString());
-        // Use WiFiClient class to create TCP connections
-        WiFiClient client;
-        if (!ControllerSettings.connectToHost(client))
-        {
-          connectionFailures++;
-          strcpy_P(log, PSTR("TELNT: connection failed"));
-          addLog(LOG_LEVEL_ERROR, log);
-          return false;
-        }
-        statusLED(true);
-        if (connectionFailures)
-          connectionFailures--;
+        LoadControllerSettings(event->ControllerIndex, ControllerSettings);
+        C003_DelayHandler.configureControllerSettings(ControllerSettings);
+        break;
+      }
 
+    case CPLUGIN_PROTOCOL_SEND:
+      {
         // We now create a URI for the request
         String url = F("variableset ");
         url += event->idx;
         url += ",";
         url += formatUserVarNoCheck(event, 0);
         url += "\n";
-
-        // strcpy_P(log, PSTR("TELNT: Sending enter"));
-        // addLog(LOG_LEVEL_ERROR, log);
-        client.print(" \n");
-
-        unsigned long timer = millis() + 200;
-        while (!client.available() && !timeOutReached(timer))
-          delay(1);
-
-        timer = millis() + 1000;
-        while (client.available() && !timeOutReached(timer) && !success)
-        {
-
-          //   String line = client.readStringUntil('\n');
-          String line;
-          safeReadStringUntil(client, line, '\n');
-
-          if (line.startsWith(F("Enter your password:")))
-          {
-            success = true;
-            strcpy_P(log, PSTR("TELNT: Password request ok"));
-            addLog(LOG_LEVEL_DEBUG, log);
-          }
-          delay(1);
-        }
-
-        strcpy_P(log, PSTR("TELNT: Sending pw"));
-        addLog(LOG_LEVEL_DEBUG, log);
-        client.println(SecuritySettings.ControllerPassword[event->ControllerIndex]);
-        delay(100);
-        while (client.available())
-          client.read();
-
-        strcpy_P(log, PSTR("TELNT: Sending cmd"));
-        addLog(LOG_LEVEL_DEBUG, log);
-        client.print(url);
-        delay(10);
-        while (client.available())
-          client.read();
-
-        strcpy_P(log, PSTR("TELNT: closing connection"));
-        addLog(LOG_LEVEL_DEBUG, log);
-
-        client.stop();
+        success = C003_DelayHandler.addToQueue(C003_queue_element(event->ControllerIndex, url));
+        scheduleNextDelayQueue(TIMER_C003_DELAY_QUEUE, C003_DelayHandler.getNextScheduleTime());
 
         break;
       }
@@ -108,4 +55,67 @@ boolean CPlugin_003(byte function, struct EventStruct *event, String& string)
   }
   return success;
 }
+
+bool do_process_c003_delay_queue(int controller_number, const C003_queue_element& element, ControllerSettingsStruct& ControllerSettings) {
+  boolean success = false;
+  char log[80];
+  addLog(LOG_LEVEL_DEBUG, String(F("TELNT : connecting to ")) + ControllerSettings.getHostPortString());
+  // Use WiFiClient class to create TCP connections
+  WiFiClient client;
+  if (!ControllerSettings.connectToHost(client))
+  {
+    connectionFailures++;
+    strcpy_P(log, PSTR("TELNT: connection failed"));
+    addLog(LOG_LEVEL_ERROR, log);
+    return success;
+  }
+  statusLED(true);
+  if (connectionFailures)
+    connectionFailures--;
+
+  // strcpy_P(log, PSTR("TELNT: Sending enter"));
+  // addLog(LOG_LEVEL_ERROR, log);
+  client.print(" \n");
+
+  unsigned long timer = millis() + 200;
+  while (!client.available() && !timeOutReached(timer))
+    delay(1);
+
+  timer = millis() + 1000;
+  while (client_available(client) && !timeOutReached(timer) && !success)
+  {
+
+    //   String line = client.readStringUntil('\n');
+    String line;
+    safeReadStringUntil(client, line, '\n');
+
+    if (line.startsWith(F("Enter your password:")))
+    {
+      success = true;
+      strcpy_P(log, PSTR("TELNT: Password request ok"));
+      addLog(LOG_LEVEL_DEBUG, log);
+    }
+    delay(1);
+  }
+
+  strcpy_P(log, PSTR("TELNT: Sending pw"));
+  addLog(LOG_LEVEL_DEBUG, log);
+  client.println(SecuritySettings.ControllerPassword[element.controller_idx]);
+  delay(100);
+  while (client.available())
+    client.read();
+
+  strcpy_P(log, PSTR("TELNT: Sending cmd"));
+  addLog(LOG_LEVEL_DEBUG, log);
+  client.print(element.txt);
+  delay(10);
+  while (client_available(client))
+    client.read();
+
+  strcpy_P(log, PSTR("TELNT: closing connection"));
+  addLog(LOG_LEVEL_DEBUG, log);
+
+  client.stop();
+  return success;
+}
 #endif

src/_C004.ino

@@ -30,6 +30,14 @@ boolean CPlugin_004(byte function, struct EventStruct *event, String& string)
         break;
       }
 
+    case CPLUGIN_INIT:
+      {
+        ControllerSettingsStruct ControllerSettings;
+        LoadControllerSettings(event->ControllerIndex, ControllerSettings);
+        C004_DelayHandler.configureControllerSettings(ControllerSettings);
+        break;
+      }
+
     case CPLUGIN_GET_PROTOCOL_DISPLAY_NAME:
       {
         success = true;
@@ -48,92 +56,44 @@ boolean CPlugin_004(byte function, struct EventStruct *event, String& string)
 
     case CPLUGIN_PROTOCOL_SEND:
       {
-        ControllerSettingsStruct ControllerSettings;
-        LoadControllerSettings(event->ControllerIndex, (byte*)&ControllerSettings, sizeof(ControllerSettings));
+        success = C004_DelayHandler.addToQueue(C004_queue_element(event));
+        scheduleNextDelayQueue(TIMER_C004_DELAY_QUEUE, C004_DelayHandler.getNextScheduleTime());
 
-        // boolean success = false;
-        addLog(LOG_LEVEL_DEBUG, String(F("HTTP : connecting to "))+ControllerSettings.getHostPortString());
-        char log[80];
-        // Use WiFiClient class to create TCP connections
-        WiFiClient client;
-        if (!ControllerSettings.connectToHost(client))
-        {
-          connectionFailures++;
-          if (loglevelActiveFor(LOG_LEVEL_ERROR)) {
-            strcpy_P(log, PSTR("HTTP : connection failed"));
-            addLog(LOG_LEVEL_ERROR, log);
-          }
-          return false;
-        }
-        statusLED(true);
-        if (connectionFailures)
-          connectionFailures--;
-
-        String postDataStr = F("api_key=");
-        postDataStr += SecuritySettings.ControllerPassword[event->ControllerIndex]; // used for API key
-
-        byte valueCount = getValueCountFromSensorType(event->sensorType);
-        for (byte x = 0; x < valueCount; x++)
-        {
-          postDataStr += F("&field");
-          postDataStr += event->idx + x;
-          postDataStr += "=";
-          postDataStr += formatUserVarNoCheck(event, x);
-        }
-        String hostName = F("api.thingspeak.com"); // PM_CZ: HTTP requests must contain host headers.
-        if (ControllerSettings.UseDNS)
-          hostName = ControllerSettings.HostName;
-
-        String postStr = F("POST /update HTTP/1.1\r\n");
-        postStr += F("Host: ");
-        postStr += hostName;
-        postStr += F("\r\n");
-        postStr += F("Connection: close\r\n");
-
-        postStr += F("Content-Type: application/x-www-form-urlencoded\r\n");
-        postStr += F("Content-Length: ");
-        postStr += postDataStr.length();
-        postStr += F("\r\n\r\n");
-        postStr += postDataStr;
-
-        // This will send the request to the server
-        client.print(postStr);
-
-        unsigned long timer = millis() + 200;
-        while (!client.available() && !timeOutReached(timer))
-          delay(1);
-
-        // Read all the lines of the reply from server and print them to Serial
-        while (client.available()) {
-          //   String line = client.readStringUntil('\n');
-          String line;
-          safeReadStringUntil(client, line, '\n');
-
-          if (loglevelActiveFor(LOG_LEVEL_DEBUG_MORE)) {
-            line.toCharArray(log, 80);
-            addLog(LOG_LEVEL_DEBUG_MORE, log);
-          }
-          if (line.substring(0, 15) == F("HTTP/1.1 200 OK"))
-          {
-            if (loglevelActiveFor(LOG_LEVEL_DEBUG)) {
-              strcpy_P(log, PSTR("HTTP : Success!"));
-              addLog(LOG_LEVEL_DEBUG, log);
-            }
-            success = true;
-          }
-          delay(1);
-        }
-        if (loglevelActiveFor(LOG_LEVEL_DEBUG)) {
-          strcpy_P(log, PSTR("HTTP : closing connection"));
-          addLog(LOG_LEVEL_DEBUG, log);
-        }
-
-        client.flush();
-        client.stop();
         break;
       }
 
   }
   return success;
 }
+
+bool do_process_c004_delay_queue(int controller_number, const C004_queue_element& element, ControllerSettingsStruct& ControllerSettings) {
+  WiFiClient client;
+  if (!try_connect_host(controller_number, client, ControllerSettings))
+    return false;
+
+  String postDataStr = F("api_key=");
+  postDataStr += SecuritySettings.ControllerPassword[element.controller_idx]; // used for API key
+
+  byte valueCount = getValueCountFromSensorType(element.sensorType);
+  for (byte x = 0; x < valueCount; x++)
+  {
+    postDataStr += F("&field");
+    postDataStr += element.idx + x;
+    postDataStr += "=";
+    postDataStr += formatUserVarNoCheck(element.TaskIndex, x);
+  }
+  String hostName = F("api.thingspeak.com"); // PM_CZ: HTTP requests must contain host headers.
+  if (ControllerSettings.UseDNS)
+    hostName = ControllerSettings.HostName;
+
+  String postStr = do_create_http_request(
+    hostName, F("POST"),
+    F("/update"), // uri
+    "",           // auth_header
+    F("Content-Type: application/x-www-form-urlencoded\r\n"),
+    postDataStr.length());
+  postStr += postDataStr;
+
+  return send_via_http(controller_number, client, postStr, ControllerSettings.MustCheckReply);
+}
 #endif

src/_C005.ino

@@ -31,6 +31,14 @@ boolean CPlugin_005(byte function, struct EventStruct *event, String& string)
         break;
       }
 
+    case CPLUGIN_INIT:
+      {
+        ControllerSettingsStruct ControllerSettings;
+        LoadControllerSettings(event->ControllerIndex, ControllerSettings);
+        MQTTDelayHandler.configureControllerSettings(ControllerSettings);
+        break;
+      }
+
     case CPLUGIN_PROTOCOL_TEMPLATE:
       {
         event->String1 = F("/%sysname%/#");
@@ -85,14 +93,14 @@ boolean CPlugin_005(byte function, struct EventStruct *event, String& string)
     case CPLUGIN_PROTOCOL_SEND:
       {
         ControllerSettingsStruct ControllerSettings;
-        LoadControllerSettings(event->ControllerIndex, (byte*)&ControllerSettings, sizeof(ControllerSettings));
+        LoadControllerSettings(event->ControllerIndex, ControllerSettings);
         if (!ControllerSettings.checkHostReachable(true)) {
             success = false;
             break;
         }
         statusLED(true);
 
-        if (ExtraTaskSettings.TaskDeviceValueNames[0][0] == 0)
+        if (ExtraTaskSettings.TaskIndex != event->TaskIndex)
           PluginCall(PLUGIN_GET_DEVICEVALUENAMES, event, dummyString);
 
         String pubname = ControllerSettings.Publish;

src/_C006.ino

@@ -31,6 +31,14 @@ boolean CPlugin_006(byte function, struct EventStruct *event, String& string)
         break;
       }
 
+    case CPLUGIN_INIT:
+      {
+        ControllerSettingsStruct ControllerSettings;
+        LoadControllerSettings(event->ControllerIndex, ControllerSettings);
+        MQTTDelayHandler.configureControllerSettings(ControllerSettings);
+        break;
+      }
+
     case CPLUGIN_PROTOCOL_TEMPLATE:
       {
         event->String1 = F("/Home/#");
@@ -83,11 +91,11 @@ boolean CPlugin_006(byte function, struct EventStruct *event, String& string)
           break;
         }
         ControllerSettingsStruct ControllerSettings;
-        LoadControllerSettings(event->ControllerIndex, (byte*)&ControllerSettings, sizeof(ControllerSettings));
+        LoadControllerSettings(event->ControllerIndex, ControllerSettings);
 
         statusLED(true);
 
-        if (ExtraTaskSettings.TaskDeviceValueNames[0][0] == 0)
+        if (ExtraTaskSettings.TaskIndex != event->TaskIndex)
           PluginCall(PLUGIN_GET_DEVICEVALUENAMES, event, dummyString);
 
         String pubname = ControllerSettings.Publish;

src/_C007.ino

@@ -30,99 +30,55 @@ boolean CPlugin_007(byte function, struct EventStruct *event, String& string)
         break;
       }
 
+    case CPLUGIN_INIT:
+      {
+        ControllerSettingsStruct ControllerSettings;
+        LoadControllerSettings(event->ControllerIndex, ControllerSettings);
+        C007_DelayHandler.configureControllerSettings(ControllerSettings);
+        break;
+      }
+
     case CPLUGIN_PROTOCOL_SEND:
       {
-        if (!WiFiConnected(100)) {
-          success = false;
-          break;
-        }
         const byte valueCount = getValueCountFromSensorType(event->sensorType);
         if (valueCount == 0 || valueCount > 3) {
           addLog(LOG_LEVEL_ERROR, F("emoncms : Unknown sensortype or too many sensor values"));
           break;
         }
-
-        ControllerSettingsStruct ControllerSettings;
-        LoadControllerSettings(event->ControllerIndex, (byte*)&ControllerSettings, sizeof(ControllerSettings));
-
-        // boolean success = false;
-        addLog(LOG_LEVEL_DEBUG, String(F("HTTP : connecting to "))+ControllerSettings.getHostPortString());
-        char log[80];
-        // Use WiFiClient class to create TCP connections
-        WiFiClient client;
-        if (!ControllerSettings.connectToHost(client))
-        {
-          connectionFailures++;
-          strcpy_P(log, PSTR("HTTP : connection failed"));
-          addLog(LOG_LEVEL_ERROR, log);
-          return false;
-        }
-        statusLED(true);
-        if (connectionFailures)
-          connectionFailures--;
-
-        String postDataStr = F("GET /emoncms/input/post.json?node=");
-
-        postDataStr += Settings.Unit;
-        postDataStr += F("&json=");
-
-        for (byte i = 0; i < valueCount; ++i) {
-          postDataStr += (i == 0) ? F("{") : F(",");
-          postDataStr += F("field");
-          postDataStr += event->idx + i;
-          postDataStr += ":";
-          postDataStr += formatUserVarNoCheck(event, i);
-        }
-        postDataStr += "}";
-        postDataStr += F("&apikey=");
-        postDataStr += SecuritySettings.ControllerPassword[event->ControllerIndex]; // "0UDNN17RW6XAS2E5" // api key
-
-        String postStr = F(" HTTP/1.1\r\n");
-        postStr += F("Host: ");
-        postStr += ControllerSettings.getHost();
-        postStr += F("\r\n");
-        postStr += F("Connection: close\r\n");
-        postStr += F("\r\n");
-
-        postDataStr += postStr;
-
-        if (Settings.SerialLogLevel >= LOG_LEVEL_DEBUG_MORE)
-          Serial.println(postDataStr);
-
-        // This will send the request to the server
-        client.print(postDataStr);
-
-        unsigned long timer = millis() + 200;
-        while (!client.available() && !timeOutReached(timer))
-          delay(1);
-
-        // Read all the lines of the reply from server and print them to Serial
-        while (client.available()) {
-          //   String line = client.readStringUntil('\n');
-          String line;
-          safeReadStringUntil(client, line, '\n');
-
-          if (loglevelActiveFor(LOG_LEVEL_DEBUG_MORE)) {
-            line.toCharArray(log, 80);
-            addLog(LOG_LEVEL_DEBUG_MORE, log);
-          }
-          if (line.substring(0, 15) == F("HTTP/1.1 200 OK"))
-          {
-            strcpy_P(log, PSTR("HTTP : Success!"));
-            addLog(LOG_LEVEL_DEBUG, log);
-            success = true;
-          }
-          delay(1);
-        }
-        strcpy_P(log, PSTR("HTTP : closing connection"));
-        addLog(LOG_LEVEL_DEBUG, log);
-
-        client.flush();
-        client.stop();
+        success = C007_DelayHandler.addToQueue(C007_queue_element(event));
+        scheduleNextDelayQueue(TIMER_C007_DELAY_QUEUE, C007_DelayHandler.getNextScheduleTime());
         break;
       }
 
   }
   return success;
 }
+
+bool do_process_c007_delay_queue(int controller_number, const C007_queue_element& element, ControllerSettingsStruct& ControllerSettings) {
+  WiFiClient client;
+  if (!try_connect_host(controller_number, client, ControllerSettings))
+    return false;
+
+  String url = F("/emoncms/input/post.json?node=");
+  url += Settings.Unit;
+  url += F("&json=");
+  const byte valueCount = getValueCountFromSensorType(element.sensorType);
+  for (byte i = 0; i < valueCount; ++i) {
+    url += (i == 0) ? F("{") : F(",");
+    url += F("field");
+    url += element.idx + i;
+    url += ":";
+    url += formatUserVarNoCheck(element.TaskIndex, i);
+  }
+  url += "}";
+  url += F("&apikey=");
+  url += SecuritySettings.ControllerPassword[element.controller_idx]; // "0UDNN17RW6XAS2E5" // api key
+
+  if (Settings.SerialLogLevel >= LOG_LEVEL_DEBUG_MORE)
+    Serial.println(url);
+
+  return send_via_http(controller_number, client,
+    create_http_get_request(controller_number, ControllerSettings, url),
+    ControllerSettings.MustCheckReply);
+}
 #endif

src/_C008.ino

@@ -32,6 +32,14 @@ boolean CPlugin_008(byte function, struct EventStruct *event, String& string)
         break;
       }
 
+    case CPLUGIN_INIT:
+      {
+        ControllerSettingsStruct ControllerSettings;
+        LoadControllerSettings(event->ControllerIndex, ControllerSettings);
+        C008_DelayHandler.configureControllerSettings(ControllerSettings);
+        break;
+      }
+
     case CPLUGIN_PROTOCOL_TEMPLATE:
       {
         event->String1 = "";
@@ -41,21 +49,31 @@ boolean CPlugin_008(byte function, struct EventStruct *event, String& string)
 
     case CPLUGIN_PROTOCOL_SEND:
       {
+        // Collect the values at the same run, to make sure all are from the same sample
         byte valueCount = getValueCountFromSensorType(event->sensorType);
+        C008_queue_element element(event, valueCount);
+        if (ExtraTaskSettings.TaskIndex != event->TaskIndex)
+          PluginCall(PLUGIN_GET_DEVICEVALUENAMES, event, dummyString);
+
+        ControllerSettingsStruct ControllerSettings;
+        LoadControllerSettings(event->ControllerIndex, ControllerSettings);
+
         for (byte x = 0; x < valueCount; x++)
         {
           bool isvalid;
           String formattedValue = formatUserVar(event, x, isvalid);
-          if (isvalid)
-            HTTPSend(event, x, formattedValue);
-          if (valueCount > 1)
-          {
-            delayBackground(Settings.MessageDelay);
-            // unsigned long timer = millis() + Settings.MessageDelay;
-            // while (!timeOutReached(timer))
-            //   backgroundtasks();
+          if (isvalid) {
+            element.txt[x] = "/";
+            element.txt[x] += ControllerSettings.Publish;
+            parseControllerVariables(element.txt[x], event, true);
+
+            element.txt[x].replace(F("%valname%"), URLEncode(ExtraTaskSettings.TaskDeviceValueNames[x]));
+            element.txt[x].replace(F("%value%"), formattedValue);
+            addLog(LOG_LEVEL_DEBUG_MORE, element.txt[x]);
           }
         }
+        success = C008_DelayHandler.addToQueue(element);
+        scheduleNextDelayQueue(TIMER_C008_DELAY_QUEUE, C008_DelayHandler.getNextScheduleTime());
         break;
       }
 
@@ -63,90 +81,23 @@ boolean CPlugin_008(byte function, struct EventStruct *event, String& string)
   return success;
 }
 
-
 //********************************************************************************
 // Generic HTTP get request
 //********************************************************************************
-boolean HTTPSend(struct EventStruct *event, byte varIndex, const String& formattedValue)
-{
-  if (!WiFiConnected(100)) {
-    return false;
-  }
-  ControllerSettingsStruct ControllerSettings;
-  LoadControllerSettings(event->ControllerIndex, (byte*)&ControllerSettings, sizeof(ControllerSettings));
-
-  String authHeader = "";
-  if ((SecuritySettings.ControllerUser[event->ControllerIndex][0] != 0) && (SecuritySettings.ControllerPassword[event->ControllerIndex][0] != 0))
-  {
-    base64 encoder;
-    String auth = SecuritySettings.ControllerUser[event->ControllerIndex];
-    auth += ":";
-    auth += SecuritySettings.ControllerPassword[event->ControllerIndex];
-    authHeader = F("Authorization: Basic ");
-    authHeader += encoder.encode(auth) + " \r\n";
+bool do_process_c008_delay_queue(int controller_number, const C008_queue_element& element, ControllerSettingsStruct& ControllerSettings) {
+  while (element.txt[element.valuesSent] == "") {
+    // A non valid value, which we are not going to send.
+    // Increase sent counter until a valid value is found.
+    if (element.checkDone(true))
+      return true;
   }
 
-  // boolean success = false;
-  addLog(LOG_LEVEL_DEBUG, String(F("HTTP : connecting to "))+ControllerSettings.getHostPortString());
-
-  // Use WiFiClient class to create TCP connections
   WiFiClient client;
-  if (!ControllerSettings.connectToHost(client))
-  {
-    connectionFailures++;
-    addLog(LOG_LEVEL_ERROR, F("HTTP : connection failed"));
+  if (!try_connect_host(controller_number, client, ControllerSettings))
     return false;
-  }
-  statusLED(true);
-  if (connectionFailures)
-    connectionFailures--;
 
-  if (ExtraTaskSettings.TaskDeviceValueNames[0][0] == 0)
-    PluginCall(PLUGIN_GET_DEVICEVALUENAMES, event, dummyString);
-
-  String url = "/";
-  url += ControllerSettings.Publish;
-  parseControllerVariables(url, event, true);
-
-  url.replace(F("%valname%"), URLEncode(ExtraTaskSettings.TaskDeviceValueNames[varIndex]));
-  url.replace(F("%value%"), formattedValue);
-
-  // url.toCharArray(log, 80);
-  addLog(LOG_LEVEL_DEBUG_MORE, url);
-
-  // This will send the request to the server
-  client.print(String(F("GET ")) + url + F(" HTTP/1.1\r\n") +
-               F("Host: ") + ControllerSettings.getHost() + F("\r\n") + authHeader +
-               F("Connection: close\r\n\r\n"));
-
-  unsigned long timer = millis() + 200;
-  while (!client.available() && !timeOutReached(timer))
-    yield();
-
-  // Read all the lines of the reply from server and print them to Serial
-  while (client.available()) {
-    // String line = client.readStringUntil('\n');
-    String line;
-    safeReadStringUntil(client, line, '\n');
-
-    // line.toCharArray(log, 80);
-    addLog(LOG_LEVEL_DEBUG_MORE, line);
-    if (line.startsWith(F("HTTP/1.1 200 OK")))
-    {
-      // strcpy_P(log, PSTR("HTTP : Success!"));
-      // addLog(LOG_LEVEL_DEBUG, log);
-      addLog(LOG_LEVEL_DEBUG, F("HTTP : Success!"));
-      // success = true;
-    }
-    delay(1);
-  }
-  // strcpy_P(log, PSTR("HTTP : closing connection"));
-  // addLog(LOG_LEVEL_DEBUG, log);
-  addLog(LOG_LEVEL_DEBUG, F("HTTP : closing connection"));
-
-  client.flush();
-  client.stop();
-
-  return(true);
+  String request = create_http_request_auth(controller_number, element.controller_idx, ControllerSettings, F("GET"), element.txt[element.valuesSent]);
+  return element.checkDone(send_via_http(controller_number, client, request, ControllerSettings.MustCheckReply));
 }
+
 #endif

src/_C009.ino

@@ -24,7 +24,7 @@
  - removed obsolete http get url code
  - v1.04
  - added build options and node_type_id to JSON/device
- /******************************************************************************/
+ ******************************************************************************/
 
 #define CPLUGIN_009
 #define CPLUGIN_ID_009         9
@@ -57,139 +57,86 @@ boolean CPlugin_009(byte function, struct EventStruct *event, String& string)
 
     case CPLUGIN_PROTOCOL_SEND:
       {
-        if (!WiFiConnected(100)) {
-          success = false;
-          break;
-        }
-        if (ExtraTaskSettings.TaskDeviceValueNames[0][0] == 0)
+        byte valueCount = getValueCountFromSensorType(event->sensorType);
+        C009_queue_element element(event);
+        if (ExtraTaskSettings.TaskIndex != event->TaskIndex)
           PluginCall(PLUGIN_GET_DEVICEVALUENAMES, event, dummyString);
 
-        // We now create a URI for the request
-        String url = F("/ESPEasy");
+        ControllerSettingsStruct ControllerSettings;
+        LoadControllerSettings(event->ControllerIndex, ControllerSettings);
 
-        // Create json root object
-        DynamicJsonBuffer jsonBuffer;
-        JsonObject& root = jsonBuffer.createObject();
-        root[F("module")] = String(F("ESPEasy"));
-        root[F("version")] = String(F("1.04"));
-
-        // Create nested objects
-        JsonObject& data = root.createNestedObject(String(F("data")));
-        JsonObject& ESP = data.createNestedObject(String(F("ESP")));
-        ESP[F("name")] = Settings.Name;
-        ESP[F("unit")] = Settings.Unit;
-        ESP[F("version")] = Settings.Version;
-        ESP[F("build")] = Settings.Build;
-        ESP[F("build_notes")] = String(F(BUILD_NOTES));
-        ESP[F("build_git")] = String(F(BUILD_GIT));
-        ESP[F("node_type_id")] = NODE_TYPE_ID;
-        ESP[F("sleep")] = Settings.deepSleep;
-
-        // embed IP, important if there is NAT/PAT
-        // char ipStr[20];
-        // IPAddress ip = WiFi.localIP();
-        // sprintf_P(ipStr, PSTR("%u.%u.%u.%u"), ip[0], ip[1], ip[2], ip[3]);
-        ESP[F("ip")] = WiFi.localIP().toString();
-
-        // Create nested SENSOR json object
-        JsonObject& SENSOR = data.createNestedObject(String(F("SENSOR")));
-        byte valueCount = getValueCountFromSensorType(event->sensorType);
-        // char itemNames[valueCount][2];
         for (byte x = 0; x < valueCount; x++)
         {
-          // Each sensor value get an own object (0..n)
-          // sprintf(itemNames[x],"%d",x);
-          JsonObject& val = SENSOR.createNestedObject(String(x));
-          val[F("deviceName")] = getTaskDeviceName(event->TaskIndex);
-          val[F("valueName")]  = ExtraTaskSettings.TaskDeviceValueNames[x];
-          val[F("type")]       = event->sensorType;
-          val[F("value")]      = formatUserVarNoCheck(event, x);
+          element.txt[x] = formatUserVarNoCheck(event, x);
+          element.valueNames[x] = ExtraTaskSettings.TaskDeviceValueNames[x];
         }
-
-        // Create json buffer
-        // char buffer[root.measureLength() +1];
-        // root.printTo(buffer, sizeof(buffer));
-        String jsonString;
-        root.printTo(jsonString);
-        // Push data to server
-        FHEMHTTPsend(url, jsonString, event->ControllerIndex);
+        success = C009_DelayHandler.addToQueue(element);
+        scheduleNextDelayQueue(TIMER_C009_DELAY_QUEUE, C009_DelayHandler.getNextScheduleTime());
         break;
       }
   }
   return success;
 }
 
-
-//********************************************************************************
-// FHEM HTTP request
-//********************************************************************************
-//TODO: create a generic HTTPSend function that we use in all the controllers. lots of code duplication here
-void FHEMHTTPsend(String & url, String & buffer, byte index)
-{
-  ControllerSettingsStruct ControllerSettings;
-  LoadControllerSettings(index, (byte*)&ControllerSettings, sizeof(ControllerSettings));
-
-  // boolean success = false;
-
-  String authHeader = "";
-  if ((SecuritySettings.ControllerUser[index][0] != 0) && (SecuritySettings.ControllerPassword[index][0] != 0)) {
-    base64 encoder;
-    String auth = SecuritySettings.ControllerUser[index];
-    auth += ":";
-    auth += SecuritySettings.ControllerPassword[index];
-    authHeader = String(F("Authorization: Basic ")) + encoder.encode(auth) + " \r\n";
-  }
-
-  addLog(LOG_LEVEL_DEBUG, String(F("HTTP : connecting to "))+ControllerSettings.getHostPortString());
-
-  // Use WiFiClient class to create TCP connections
+/*********************************************************************************************\
+ * FHEM HTTP request
+\*********************************************************************************************/
+bool do_process_c009_delay_queue(int controller_number, const C009_queue_element& element, ControllerSettingsStruct& ControllerSettings) {
   WiFiClient client;
-  if (!ControllerSettings.connectToHost(client)) {
-    connectionFailures++;
-    // strcpy_P(log, PSTR("HTTP : connection failed"));
-    addLog(LOG_LEVEL_ERROR, F("HTTP : connection failed"));
-    return;
+  if (!try_connect_host(controller_number, client, ControllerSettings))
+    return false;
+
+  // Create json root object
+  DynamicJsonBuffer jsonBuffer;
+  JsonObject& root = jsonBuffer.createObject();
+  root[F("module")] = String(F("ESPEasy"));
+  root[F("version")] = String(F("1.04"));
+
+  // Create nested objects
+  JsonObject& data = root.createNestedObject(String(F("data")));
+  JsonObject& ESP = data.createNestedObject(String(F("ESP")));
+  ESP[F("name")] = Settings.Name;
+  ESP[F("unit")] = Settings.Unit;
+  ESP[F("version")] = Settings.Version;
+  ESP[F("build")] = Settings.Build;
+  ESP[F("build_notes")] = String(F(BUILD_NOTES));
+  ESP[F("build_git")] = String(F(BUILD_GIT));
+  ESP[F("node_type_id")] = NODE_TYPE_ID;
+  ESP[F("sleep")] = Settings.deepSleep;
+
+  // embed IP, important if there is NAT/PAT
+  // char ipStr[20];
+  // IPAddress ip = WiFi.localIP();
+  // sprintf_P(ipStr, PSTR("%u.%u.%u.%u"), ip[0], ip[1], ip[2], ip[3]);
+  ESP[F("ip")] = WiFi.localIP().toString();
+
+  // Create nested SENSOR json object
+  JsonObject& SENSOR = data.createNestedObject(String(F("SENSOR")));
+  byte valueCount = getValueCountFromSensorType(element.sensorType);
+  // char itemNames[valueCount][2];
+  for (byte x = 0; x < valueCount; x++)
+  {
+    // Each sensor value get an own object (0..n)
+    // sprintf(itemNames[x],"%d",x);
+    JsonObject& val = SENSOR.createNestedObject(String(x));
+    val[F("deviceName")] = getTaskDeviceName(element.TaskIndex);
+    val[F("valueName")]  = element.valueNames[x];
+    val[F("type")]       = element.sensorType;
+    val[F("value")]      = element.txt[x];
   }
 
-  statusLED(true);
-  if (connectionFailures)
-    connectionFailures--;
+  // Create json buffer
+  String jsonString;
+  root.printTo(jsonString);
 
-  // This will send the request to the server
-  int len = buffer.length();
-  client.print(String("POST ") + url + F(" HTTP/1.1\r\n") +
-              F("Content-Length: ")+ len + F("\r\n") +
-              F("Host: ") + ControllerSettings.getHost() + F("\r\n") + authHeader +
-              F("Connection: close\r\n\r\n")
-              + buffer);
+  // We now create a URI for the request
+  String url = F("/ESPEasy");
 
-  unsigned long timer = millis() + 200;
-  while (!client.available() && !timeOutReached(timer))
-    yield();
+  String request = create_http_request_auth(
+      controller_number, element.controller_idx, ControllerSettings,
+      F("POST"), url, jsonString.length());
+  request += jsonString;
 
-  // Read all the lines of the reply from server and print them to Serial
-  while (client.available()) {
-    // String line = client.readStringUntil('\n');
-    String line;
-    safeReadStringUntil(client, line, '\n');
-
-    // String helper = line;
-    // line.toCharArray(log, 80);
-    addLog(LOG_LEVEL_DEBUG_MORE, line);
-
-    if (line.startsWith(F("HTTP/1.1 200 OK"))) {
-      // strcpy_P(log, PSTR("HTTP : Success"));
-      addLog(LOG_LEVEL_DEBUG_MORE, F("HTTP : Success"));
-      // success = true;
-    }
-    else if (line.startsWith(F("HTTP/1.1 4"))) {
-      addLog(LOG_LEVEL_ERROR, String(F("HTTP : Error: "))+line);
-    }
-    yield();
-  }
-  // strcpy_P(log, PSTR("HTTP : closing connection"));
-  addLog(LOG_LEVEL_DEBUG, F("HTTP : closing connection"));
-  client.flush();
-  client.stop();
+  return send_via_http(controller_number, client, request, ControllerSettings.MustCheckReply);
 }
 #endif

src/_C010.ino

@@ -41,20 +41,32 @@ boolean CPlugin_010(byte function, struct EventStruct *event, String& string)
     case CPLUGIN_PROTOCOL_SEND:
       {
         byte valueCount = getValueCountFromSensorType(event->sensorType);
+        C010_queue_element element(event, valueCount);
+        if (ExtraTaskSettings.TaskIndex != event->TaskIndex)
+          PluginCall(PLUGIN_GET_DEVICEVALUENAMES, event, dummyString);
+
+        ControllerSettingsStruct ControllerSettings;
+        LoadControllerSettings(event->ControllerIndex, ControllerSettings);
+
         for (byte x = 0; x < valueCount; x++)
         {
           bool isvalid;
           String formattedValue = formatUserVar(event, x, isvalid);
-          if (isvalid)
-            C010_Send(event, x, formattedValue);
-          if (valueCount > 1)
-          {
-            delayBackground(Settings.MessageDelay);
-            // unsigned long timer = millis() + Settings.MessageDelay;
-            // while (!timeOutReached(timer))
-            //   backgroundtasks();
+          if (isvalid) {
+            element.txt[x] = "";
+            element.txt[x] += ControllerSettings.Publish;
+            parseControllerVariables(element.txt[x], event, false);
+            element.txt[x].replace(F("%valname%"), ExtraTaskSettings.TaskDeviceValueNames[x]);
+            element.txt[x].replace(F("%value%"), formattedValue);
+            if (loglevelActiveFor(LOG_LEVEL_DEBUG_MORE)) {
+              char log[80];
+              element.txt[x].toCharArray(log, 80);
+              addLog(LOG_LEVEL_DEBUG_MORE, log);
+            }
           }
         }
+        success = C010_DelayHandler.addToQueue(element);
+        scheduleNextDelayQueue(TIMER_C010_DELAY_QUEUE, C010_DelayHandler.getNextScheduleTime());
         break;
       }
 
@@ -66,34 +78,25 @@ boolean CPlugin_010(byte function, struct EventStruct *event, String& string)
 //********************************************************************************
 // Generic UDP message
 //********************************************************************************
-void C010_Send(struct EventStruct *event, byte varIndex, const String& formattedValue)
-{
-  ControllerSettingsStruct ControllerSettings;
-  LoadControllerSettings(event->ControllerIndex, (byte*)&ControllerSettings, sizeof(ControllerSettings));
-
-  // boolean success = false;
-  addLog(LOG_LEVEL_DEBUG, String(F("UDP  : sending to ")) + ControllerSettings.getHostPortString());
-  statusLED(true);
-
-  if (ExtraTaskSettings.TaskDeviceValueNames[0][0] == 0)
-    PluginCall(PLUGIN_GET_DEVICEVALUENAMES, event, dummyString);
-
-  String msg = "";
-  msg += ControllerSettings.Publish;
-  parseControllerVariables(msg, event, false);
-  msg.replace(F("%valname%"), ExtraTaskSettings.TaskDeviceValueNames[varIndex]);
-  msg.replace(F("%value%"), formattedValue);
-
-  if (wifiStatus == ESPEASY_WIFI_SERVICES_INITIALIZED) {
-    ControllerSettings.beginPacket(portUDP);
-    portUDP.write((uint8_t*)msg.c_str(),msg.length());
-    portUDP.endPacket();
+bool do_process_c010_delay_queue(int controller_number, const C010_queue_element& element, ControllerSettingsStruct& ControllerSettings) {
+  while (element.txt[element.valuesSent] == "") {
+    // A non valid value, which we are not going to send.
+    // Increase sent counter until a valid value is found.
+    if (element.checkDone(true))
+      return true;
   }
+  WiFiUDP C010_portUDP;
+  if (!beginWiFiUDP_randomPort(C010_portUDP)) return false;
+  if (!try_connect_host(controller_number, C010_portUDP, ControllerSettings))
+    return false;
 
-  if (loglevelActiveFor(LOG_LEVEL_DEBUG_MORE)) {
-    char log[80];
-    msg.toCharArray(log, 80);
-    addLog(LOG_LEVEL_DEBUG_MORE, log);
-  }
+  C010_portUDP.write(
+    (uint8_t*)element.txt[element.valuesSent].c_str(),
+              element.txt[element.valuesSent].length());
+  bool reply = C010_portUDP.endPacket();
+  C010_portUDP.stop();
+  if (ControllerSettings.MustCheckReply)
+    return element.checkDone(reply);
+  return element.checkDone(true);
 }
 #endif

src/_C011.ino

@@ -117,68 +117,53 @@ boolean CPlugin_011(byte function, struct EventStruct *event, String& string)
 
     case CPLUGIN_PROTOCOL_SEND:
       {
-      	HTTPSend011(event);
+      	success = Create_schedule_HTTP_C011(event);
+        break;
       }
 
   }
   return success;
 }
 
+//********************************************************************************
+// Generic HTTP request
+//********************************************************************************
+bool do_process_c011_delay_queue(int controller_number, const C011_queue_element& element, ControllerSettingsStruct& ControllerSettings) {
+  WiFiClient client;
+  if (!try_connect_host(controller_number, client, ControllerSettings))
+    return false;
+
+  return send_via_http(controller_number, client, element.txt, ControllerSettings.MustCheckReply);
+}
+
+
 
 //********************************************************************************
-// Generic HTTP get request
+// Create request
 //********************************************************************************
-boolean HTTPSend011(struct EventStruct *event)
+boolean Create_schedule_HTTP_C011(struct EventStruct *event)
 {
+  int controller_number = CPLUGIN_ID_011;
   if (!WiFiConnected(100)) {
     return false;
   }
   ControllerSettingsStruct ControllerSettings;
-  LoadControllerSettings(event->ControllerIndex, (byte*)&ControllerSettings, sizeof(ControllerSettings));
-
-  String authHeader = "";
-  if ((SecuritySettings.ControllerUser[event->ControllerIndex][0] != 0) && (SecuritySettings.ControllerPassword[event->ControllerIndex][0] != 0))
-  {
-    base64 encoder;
-    String auth = SecuritySettings.ControllerUser[event->ControllerIndex];
-    auth += ":";
-    auth += SecuritySettings.ControllerPassword[event->ControllerIndex];
-    authHeader = F("Authorization: Basic ");
-    authHeader += encoder.encode(auth);
-    authHeader += F(" \r\n");
-  }
+  LoadControllerSettings(event->ControllerIndex, ControllerSettings);
 
   C011_ConfigStruct customConfig;
   LoadCustomControllerSettings(event->ControllerIndex,(byte*)&customConfig, sizeof(customConfig));
   customConfig.zero_last();
 
-  boolean success = false;
-  addLog(LOG_LEVEL_DEBUG, String(F("HTTP : connecting to "))+
-      ControllerSettings.getHostPortString());
-
-  // Use WiFiClient class to create TCP connections
   WiFiClient client;
-  if (!ControllerSettings.connectToHost(client))
-  {
-    connectionFailures++;
-    addLog(LOG_LEVEL_ERROR, F("HTTP : connection failed"));
+  if (!try_connect_host(controller_number, client, ControllerSettings))
     return false;
-  }
-  statusLED(true);
-  if (connectionFailures)
-    connectionFailures--;
 
-  if (ExtraTaskSettings.TaskDeviceValueNames[0][0] == 0)
+  if (ExtraTaskSettings.TaskIndex != event->TaskIndex)
     PluginCall(PLUGIN_GET_DEVICEVALUENAMES, event, dummyString);
 
-  String payload = String(customConfig.HttpMethod) + " /";
-  payload += customConfig.HttpUri;
-  payload += F(" HTTP/1.1\r\n");
-  payload += F("Host: ");
-  payload += ControllerSettings.getHostPortString();
-  payload += F("\r\n");
-  payload += authHeader;
-  payload += F("Connection: close\r\n");
+  String payload = create_http_request_auth(
+    controller_number, event->ControllerIndex, ControllerSettings,
+    String(customConfig.HttpMethod), customConfig.HttpUri);
 
   if (strlen(customConfig.HttpHeader) > 0)
     payload += customConfig.HttpHeader;
@@ -195,36 +180,9 @@ boolean HTTPSend011(struct EventStruct *event)
   }
   payload += F("\r\n");
 
-  // This will send the request to the server
-  client.print(payload);
-  addLog(LOG_LEVEL_DEBUG_MORE, payload);
-
-  unsigned long timer = millis() + 200;
-  while (!client.available() && !timeOutReached(timer))
-    yield();
-
-  // Read all the lines of the reply from server and print them to Serial
-  while (client.available()) {
-    // String line = client.readStringUntil('\n');
-    String line;
-    safeReadStringUntil(client, line, '\n');
-
-
-    // line.toCharArray(log, 80);
-    addLog(LOG_LEVEL_DEBUG_MORE, line);
-    if (line.startsWith(F("HTTP/1.1 2")))
-    {
-      addLog(LOG_LEVEL_DEBUG, F("HTTP : Success!"));
-      success = true;
-    }
-    yield();
-  }
-  addLog(LOG_LEVEL_DEBUG, F("HTTP : closing connection"));
-
-  client.flush();
-  client.stop();
-
-  return(success);
+  bool success = C011_DelayHandler.addToQueue(C011_queue_element(event->ControllerIndex, payload));
+  scheduleNextDelayQueue(TIMER_C011_DELAY_QUEUE, C011_DelayHandler.getNextScheduleTime());
+  return success;
 }
 
 // parses the string and returns only the the number of name/values we want

src/_C012.ino

@@ -34,52 +34,65 @@ boolean CPlugin_012(byte function, struct EventStruct *event, String& string)
 
      case CPLUGIN_PROTOCOL_SEND:
       {
-        if (wifiStatus != ESPEASY_WIFI_SERVICES_INITIALIZED) {
-          success = false;
-          break;
-        }
+        // Collect the values at the same run, to make sure all are from the same sample
+        byte valueCount = getValueCountFromSensorType(event->sensorType);
+        C012_queue_element element(event, valueCount);
+        if (ExtraTaskSettings.TaskIndex != event->TaskIndex)
+          PluginCall(PLUGIN_GET_DEVICEVALUENAMES, event, dummyString);
 
-        String postDataStr = F("");
-        const byte valueCount = getValueCountFromSensorType(event->sensorType);
-        success = CPlugin_012_send(event, valueCount);
+        ControllerSettingsStruct ControllerSettings;
+        LoadControllerSettings(event->ControllerIndex, ControllerSettings);
+
+        for (byte x = 0; x < valueCount; x++)
+        {
+          bool isvalid;
+          String formattedValue = formatUserVar(event, x, isvalid);
+          if (isvalid) {
+            element.txt[x] = F("update/V");
+            element.txt[x] += event->idx + x;
+            element.txt[x] += F("?value=");
+            element.txt[x] += formattedValue;
+            addLog(LOG_LEVEL_DEBUG_MORE, element.txt[x]);
+          }
+        }
+        success = C012_DelayHandler.addToQueue(element);
+        scheduleNextDelayQueue(TIMER_C012_DELAY_QUEUE, C012_DelayHandler.getNextScheduleTime());
         break;
       }
   }
   return success;
 }
 
-boolean CPlugin_012_send(struct EventStruct *event, int nrValues) {
-  String postDataStr = F("");
-  boolean success = true;
-  for (int i = 0; i < nrValues && success; ++i) {
-    postDataStr = F("update/V") ;
-    postDataStr += event->idx + i;
-    postDataStr += F("?value=");
-    postDataStr += formatUserVarNoCheck(event, i);
-    success = Blynk_get(postDataStr, event->ControllerIndex);
+//********************************************************************************
+// Process Queued Blynk request, with data set to NULL
+//********************************************************************************
+bool do_process_c012_delay_queue(int controller_number, const C012_queue_element& element, ControllerSettingsStruct& ControllerSettings) {
+  while (element.txt[element.valuesSent] == "") {
+    // A non valid value, which we are not going to send.
+    // Increase sent counter until a valid value is found.
+    if (element.checkDone(true))
+      return true;
   }
-  return success;
-}
-
-
-boolean Blynk_get(const String& command, byte controllerIndex, float *data )
-{
   if (wifiStatus != ESPEASY_WIFI_SERVICES_INITIALIZED) {
     return false;
   }
+  return element.checkDone(Blynk_get(element.txt[element.valuesSent], element.controller_idx));
+}
 
+boolean Blynk_get(const String& command, byte controllerIndex, float *data )
+{
   ControllerSettingsStruct ControllerSettings;
-  LoadControllerSettings(controllerIndex, (byte*)&ControllerSettings, sizeof(ControllerSettings));
-  // Use WiFiClient class to create TCP connections
-  WiFiClient client;
-  if ((SecuritySettings.ControllerPassword[controllerIndex][0] == 0) || !ControllerSettings.connectToHost(client))
-  {
-    connectionFailures++;
-    addLog(LOG_LEVEL_ERROR, F("Blynk : connection failed"));
+  LoadControllerSettings(controllerIndex, ControllerSettings);
+
+  if ((SecuritySettings.ControllerPassword[controllerIndex][0] == 0)) {
+    addLog(LOG_LEVEL_ERROR, F("Blynk : No password set"));
     return false;
   }
-  if (connectionFailures)
-    connectionFailures--;
+
+  WiFiClient client;
+  if (!try_connect_host(CPLUGIN_ID_012, client, ControllerSettings))
+    return false;
+
 
   // We now create a URI for the request
   char request[300] = {0};
@@ -90,58 +103,58 @@ boolean Blynk_get(const String& command, byte controllerIndex, float *data )
             ControllerSettings.getHost().c_str());
   addLog(LOG_LEVEL_DEBUG, request);
   client.print(request);
+  boolean success = !ControllerSettings.MustCheckReply;
+  if (ControllerSettings.MustCheckReply || data) {
+    unsigned long timer = millis() + 200;
+    while (!client.available() && !timeOutReached(timer))
+      yield();
 
-  unsigned long timer = millis() + 200;
-  while (!client.available() && !timeOutReached(timer))
-    yield();
+    char log[80] = {0};
 
-  boolean success = false;
-  char log[80] = {0};
-
-  // Read all the lines of the reply from server and log them
-  while (client.available()) {
-    String line;
-    safeReadStringUntil(client, line, '\n');
-    addLog(LOG_LEVEL_DEBUG_MORE, line);
-    // success ?
-    if (line.substring(0, 15) == F("HTTP/1.1 200 OK")) {
-      strcpy_P(log, PSTR("HTTP : Success"));
-      success = true;
-    }
-    else if (line.substring(0, 24) == F("HTTP/1.1 400 Bad Request")) {
-      strcpy_P(log, PSTR("HTTP : Unauthorized"));
-    }
-    else if (line.substring(0, 25) == F("HTTP/1.1 401 Unauthorized")) {
-      strcpy_P(log, PSTR("HTTP : Unauthorized"));
-    }
-    addLog(LOG_LEVEL_DEBUG, log);
-
-    // data only
-    if (data && line.startsWith("["))
-    {
-      String strValue = line;
-      byte pos = strValue.indexOf('"',2);
-      strValue = strValue.substring(2, pos);
-      strValue.trim();
-      float value = strValue.toFloat();
-      *data = value;
-      success = true;
-
-      char value_char[5] = {0};
-      strValue.toCharArray(value_char, 5);
-      sprintf_P(log, PSTR("Blynk get - %s => %s"),command.c_str(), value_char   );
+    // Read all the lines of the reply from server and log them
+    while (client_available(client)) {
+      String line;
+      safeReadStringUntil(client, line, '\n');
+      addLog(LOG_LEVEL_DEBUG_MORE, line);
+      // success ?
+      if (line.substring(0, 15) == F("HTTP/1.1 200 OK")) {
+        strcpy_P(log, PSTR("HTTP : Success"));
+        success = true;
+      }
+      else if (line.substring(0, 24) == F("HTTP/1.1 400 Bad Request")) {
+        strcpy_P(log, PSTR("HTTP : Unauthorized"));
+      }
+      else if (line.substring(0, 25) == F("HTTP/1.1 401 Unauthorized")) {
+        strcpy_P(log, PSTR("HTTP : Unauthorized"));
+      }
       addLog(LOG_LEVEL_DEBUG, log);
+
+      // data only
+      if (data && line.startsWith("["))
+      {
+        String strValue = line;
+        byte pos = strValue.indexOf('"',2);
+        strValue = strValue.substring(2, pos);
+        strValue.trim();
+        float value = strValue.toFloat();
+        *data = value;
+        success = true;
+
+        char value_char[5] = {0};
+        strValue.toCharArray(value_char, 5);
+        sprintf_P(log, PSTR("Blynk get - %s => %s"),command.c_str(), value_char   );
+        addLog(LOG_LEVEL_DEBUG, log);
+      }
+      yield();
     }
-    yield();
   }
-  strcpy_P(log, PSTR("HTTP : closing connection"));
-  addLog(LOG_LEVEL_DEBUG, log);
+  addLog(LOG_LEVEL_DEBUG, F("HTTP : closing connection (012)"));
 
   client.flush();
   client.stop();
 
-  // important - backgroudtasks - free mem
-  timer = millis() + Settings.MessageDelay;
+  // important - backgroundtasks - free mem
+  unsigned long timer = millis() + Settings.MessageDelay;
   while (!timeOutReached(timer))
               backgroundtasks();
 

src/_C013.ino

@@ -80,7 +80,7 @@ boolean CPlugin_013(byte function, struct EventStruct *event, String& string)
 
     case CPLUGIN_PROTOCOL_SEND:
       {
-        C013_Send(event, 0, UserVar[event->BaseVarIndex], 0);
+        C013_SendUDPTaskData(0, event->TaskIndex, event->TaskIndex);
         break;
       }
 
@@ -98,14 +98,6 @@ boolean CPlugin_013(byte function, struct EventStruct *event, String& string)
 //********************************************************************************
 // Generic UDP message
 //********************************************************************************
-void C013_Send(struct EventStruct *event, byte varIndex, float value, unsigned long longValue)
-{
-  ControllerSettingsStruct ControllerSettings;
-  LoadControllerSettings(event->ControllerIndex, (byte*)&ControllerSettings, sizeof(ControllerSettings));
-  statusLED(true);
-  C013_SendUDPTaskData(0, event->TaskIndex, event->TaskIndex);
-}
-
 void C013_SendUDPTaskInfo(byte destUnit, byte sourceTaskIndex, byte destTaskIndex)
 {
   if (!WiFiConnected(100)) {
@@ -121,19 +113,18 @@ void C013_SendUDPTaskInfo(byte destUnit, byte sourceTaskIndex, byte destTaskInde
   for (byte x = 0; x < VARS_PER_TASK; x++)
     strcpy(infoReply.ValueNames[x], ExtraTaskSettings.TaskDeviceValueNames[x]);
 
-  byte firstUnit = 1;
-  byte lastUnit = UNIT_MAX - 1;
   if (destUnit != 0)
   {
-    firstUnit = destUnit;
-    lastUnit = destUnit;
-  }
-  for (byte x = firstUnit; x <= lastUnit; x++)
-  {
-    if (x != Settings.Unit){
-      infoReply.destUnit = x;
-      C013_sendUDP(x, (byte*)&infoReply, sizeof(infoStruct));
-      delay(10);
+    infoReply.destUnit = destUnit;
+    C013_sendUDP(destUnit, (byte*)&infoReply, sizeof(infoStruct));
+    delay(10);
+  } else {
+    for (NodesMap::iterator it = Nodes.begin(); it != Nodes.end(); ++it) {
+      if (it->first != Settings.Unit) {
+        infoReply.destUnit = it->first;
+        C013_sendUDP(it->first, (byte*)&infoReply, sizeof(infoStruct));
+        delay(10);
+      }
     }
   }
   delay(50);
@@ -151,19 +142,18 @@ void C013_SendUDPTaskData(byte destUnit, byte sourceTaskIndex, byte destTaskInde
   for (byte x = 0; x < VARS_PER_TASK; x++)
     dataReply.Values[x] = UserVar[dataReply.sourceTaskIndex * VARS_PER_TASK + x];
 
-  byte firstUnit = 1;
-  byte lastUnit = UNIT_MAX - 1;
   if (destUnit != 0)
   {
-    firstUnit = destUnit;
-    lastUnit = destUnit;
-  }
-  for (byte x = firstUnit; x <= lastUnit; x++)
-  {
-    if (x != Settings.Unit){
-      dataReply.destUnit = x;
-      C013_sendUDP(x, (byte*) &dataReply, sizeof(dataStruct));
-      delay(10);
+    dataReply.destUnit = destUnit;
+    C013_sendUDP(destUnit, (byte*) &dataReply, sizeof(dataStruct));
+    delay(10);
+  } else {
+    for (NodesMap::iterator it = Nodes.begin(); it != Nodes.end(); ++it) {
+      if (it->first != Settings.Unit) {
+        dataReply.destUnit = it->first;
+        C013_sendUDP(it->first, (byte*) &dataReply, sizeof(dataStruct));
+        delay(10);
+      }
     }
   }
   delay(50);
@@ -177,9 +167,14 @@ void C013_sendUDP(byte unit, byte* data, byte size)
   if (!WiFiConnected(100)) {
     return;
   }
-  if (unit != 255)
-    if (Nodes[unit].ip[0] == 0)
+  NodesMap::iterator it;
+  if (unit != 255) {
+    it = Nodes.find(unit);
+    if (it == Nodes.end())
       return;
+    if (it->second.ip[0] == 0)
+      return;
+  }
   if (loglevelActiveFor(LOG_LEVEL_DEBUG_MORE)) {
     String log = F("C013 : Send UDP message to ");
     log += unit;
@@ -192,7 +187,7 @@ void C013_sendUDP(byte unit, byte* data, byte size)
   if (unit == 255)
     remoteNodeIP = {255, 255, 255, 255};
   else
-    remoteNodeIP = Nodes[unit].ip;
+    remoteNodeIP = it->second.ip;
   if (!beginWiFiUDP_randomPort(C013_portUDP)) return;
   if (C013_portUDP.beginPacket(remoteNodeIP, Settings.UDPPort) == 0) return;
   C013_portUDP.write(data, size);

src/_CPlugin_Helper.h

@@ -0,0 +1,692 @@
+#ifndef CPLUGIN_HELPER_H
+#define CPLUGIN_HELPER_H CPLUGIN_HELPER_H
+
+// These element classes should be defined as class, to be used as template.
+
+/*********************************************************************************************\
+ * MQTT_queue_element for all MQTT base controllers
+\*********************************************************************************************/
+class MQTT_queue_element {
+public:
+  MQTT_queue_element() : controller_idx(0), _retained(false) {}
+
+  MQTT_queue_element(int ctrl_idx,
+    const String& topic, const String& payload, boolean retained) :
+    controller_idx(ctrl_idx), _topic(topic), _payload(payload), _retained(retained)
+     {}
+
+  size_t getSize() const {
+    return sizeof(this) + _topic.length() + _payload.length();
+  }
+
+  int controller_idx;
+  String _topic;
+  String _payload;
+  boolean _retained;
+};
+
+/*********************************************************************************************\
+ * Simple queue element, only storing controller index and some String
+\*********************************************************************************************/
+class simple_queue_element_string_only {
+public:
+  simple_queue_element_string_only() : controller_idx(0) {}
+  simple_queue_element_string_only(int ctrl_idx, const String& req) :
+       controller_idx(ctrl_idx), txt(req) {}
+
+  size_t getSize() const {
+    return sizeof(this) + txt.length();
+  }
+
+  int controller_idx;
+  String txt;
+};
+
+
+/*********************************************************************************************\
+ * C001_queue_element for queueing requests for C001.
+\*********************************************************************************************/
+#define C001_queue_element simple_queue_element_string_only
+
+/*********************************************************************************************\
+ * C003_queue_element for queueing requests for C003 Nodo Telnet.
+\*********************************************************************************************/
+#define C003_queue_element simple_queue_element_string_only
+
+/*********************************************************************************************\
+ * C004_queue_element for queueing requests for C004 ThingSpeak.
+ *   Typical use case for Thingspeak is to only send values every N seconds/minutes.
+ *   So we just store everything needed to recreate the event when the time is ready.
+\*********************************************************************************************/
+class C004_queue_element {
+public:
+  C004_queue_element() : controller_idx(0), TaskIndex(0), idx(0), sensorType(0) {}
+  C004_queue_element(const struct EventStruct* event) :
+    controller_idx(event->ControllerIndex),
+    TaskIndex(event->TaskIndex),
+    idx(event->idx),
+    sensorType(event->sensorType) {}
+
+  size_t getSize() const {
+    return sizeof(this);
+  }
+
+  int controller_idx;
+  byte TaskIndex;
+  int idx;
+  byte sensorType;
+};
+
+/*********************************************************************************************\
+ * C007_queue_element for queueing requests for C007 Emoncms
+\*********************************************************************************************/
+class C007_queue_element {
+public:
+  C007_queue_element() : controller_idx(0), TaskIndex(0), idx(0), sensorType(0) {}
+  C007_queue_element(const struct EventStruct* event) :
+    controller_idx(event->ControllerIndex),
+    TaskIndex(event->TaskIndex),
+    idx(event->idx),
+    sensorType(event->sensorType) {}
+
+  size_t getSize() const {
+    return sizeof(this);
+  }
+
+  int controller_idx;
+  byte TaskIndex;
+  int idx;
+  byte sensorType;
+};
+
+/*********************************************************************************************\
+ * Base class for controllers that only send a single value per request and thus needs to
+ * keep track of the number of values already sent.
+\*********************************************************************************************/
+class queue_element_single_value_base {
+public:
+  queue_element_single_value_base() : controller_idx(0), TaskIndex(0), idx(0), valuesSent(0) {}
+  queue_element_single_value_base(const struct EventStruct* event, byte value_count) :
+    controller_idx(event->ControllerIndex),
+    TaskIndex(event->TaskIndex),
+    idx(event->idx),
+    valuesSent(0),
+    valueCount(value_count) {}
+
+  bool checkDone(bool succesfull) const {
+    if (succesfull) ++valuesSent;
+    return (valuesSent >= valueCount || valuesSent >= VARS_PER_TASK);
+  }
+
+  size_t getSize() const {
+    size_t total = sizeof(this);
+    for (int i = 0; i < VARS_PER_TASK; ++i) {
+      total += txt[i].length();
+    }
+    return total;
+  }
+
+  String txt[VARS_PER_TASK];
+  int controller_idx;
+  byte TaskIndex;
+  int idx;
+  mutable byte valuesSent;  // Value must be set by const function checkDone()
+  byte valueCount;
+};
+
+
+/*********************************************************************************************\
+ * C008_queue_element for queueing requests for 008: Generic HTTP
+ * Using queue_element_single_value_base
+\*********************************************************************************************/
+#define C008_queue_element queue_element_single_value_base
+
+/*********************************************************************************************\
+ * C009_queue_element for queueing requests for C009: FHEM HTTP.
+\*********************************************************************************************/
+class C009_queue_element {
+public:
+  C009_queue_element() : controller_idx(0), TaskIndex(0), idx(0), sensorType(0) {}
+  C009_queue_element(const struct EventStruct* event) :
+    controller_idx(event->ControllerIndex),
+    TaskIndex(event->TaskIndex),
+    idx(event->idx),
+    sensorType(event->sensorType) {}
+
+  size_t getSize() const {
+    size_t total = sizeof(this);
+    for (int i = 0; i < VARS_PER_TASK; ++i) {
+      total += txt[i].length();
+      total += valueNames[i].length();
+    }
+    return total;
+  }
+
+  String txt[VARS_PER_TASK];
+  String valueNames[VARS_PER_TASK];
+  int controller_idx;
+  byte TaskIndex;
+  int idx;
+  byte sensorType;
+};
+
+
+/*********************************************************************************************\
+ * C010_queue_element for queueing requests for 010: Generic UDP
+ * Using queue_element_single_value_base
+\*********************************************************************************************/
+#define C010_queue_element queue_element_single_value_base
+
+/*********************************************************************************************\
+ * C011_queue_element for queueing requests for 011: Generic HTTP Advanced
+\*********************************************************************************************/
+#define C011_queue_element simple_queue_element_string_only
+
+/*********************************************************************************************\
+ * C012_queue_element for queueing requests for 012: Blynk
+ * Using queue_element_single_value_base
+\*********************************************************************************************/
+#define C012_queue_element queue_element_single_value_base
+
+
+
+/*********************************************************************************************\
+ * ControllerDelayHandlerStruct
+\*********************************************************************************************/
+template<class T>
+struct ControllerDelayHandlerStruct {
+  ControllerDelayHandlerStruct() :
+      lastSend(0),
+      minTimeBetweenMessages(CONTROLLER_DELAY_QUEUE_DELAY_DFLT),
+      max_queue_depth(CONTROLLER_DELAY_QUEUE_DEPTH_DFLT),
+      attempt(0),
+      max_retries(CONTROLLER_DELAY_QUEUE_RETRY_DFLT),
+      delete_oldest(false),
+      must_check_reply(false) {}
+
+  void configureControllerSettings(const ControllerSettingsStruct& settings) {
+    minTimeBetweenMessages = settings.MinimalTimeBetweenMessages;
+    max_queue_depth = settings.MaxQueueDepth;
+    max_retries = settings.MaxRetry;
+    delete_oldest = settings.DeleteOldest;
+    must_check_reply = settings.MustCheckReply;
+    // Set some sound limits when not configured
+    if (max_queue_depth == 0) max_queue_depth = CONTROLLER_DELAY_QUEUE_DEPTH_DFLT;
+    if (max_retries == 0) max_retries = CONTROLLER_DELAY_QUEUE_RETRY_DFLT;
+    if (minTimeBetweenMessages == 0) minTimeBetweenMessages = CONTROLLER_DELAY_QUEUE_DELAY_DFLT;
+    // No less than 10 msec between messages.
+    if (minTimeBetweenMessages < 10) minTimeBetweenMessages = 10;
+  }
+
+  bool queueFull(const T& element) const {
+    if (sendQueue.size() >= max_queue_depth) return true;
+
+    // Number of elements is not exceeding the limit, check memory
+    int freeHeap = ESP.getFreeHeap();
+    if (freeHeap > 5000) return false; // Memory is not an issue.
+    if (loglevelActiveFor(LOG_LEVEL_DEBUG)) {
+      String log = "Controller-";
+      log += element.controller_idx +1;
+      log += " : Memory used: ";
+      log += getQueueMemorySize();
+      log += " bytes ";
+      log += sendQueue.size();
+      log += " items ";
+      log += freeHeap;
+      log += " free";
+      addLog(LOG_LEVEL_DEBUG, log);
+    }
+    return true;
+  }
+
+  // Try to add to the queue, if permitted by "delete_oldest"
+  // Return false when no item was added.
+  bool addToQueue(const T& element) {
+    if (delete_oldest) {
+      // Force add to the queue.
+      // If max buffer is reached, the oldest in the queue (first to be served) will be removed.
+      while (queueFull(element)) {
+        sendQueue.pop_front();
+      }
+      sendQueue.emplace_back(element);
+      return true;
+    }
+    if (!queueFull(element)) {
+      sendQueue.emplace_back(element);
+      return true;
+    }
+    if (loglevelActiveFor(LOG_LEVEL_DEBUG)) {
+      String log = get_formatted_Controller_number(element.controller_idx);
+      log += " : queue full";
+      addLog(LOG_LEVEL_DEBUG, log);
+    }
+    return false;
+  }
+
+  // Get the next element.
+  // Remove front element when max_retries is reached.
+  bool getNext(T& element) {
+    if (sendQueue.empty()) return false;
+    if (attempt > max_retries) {
+      sendQueue.pop_front();
+      attempt = 0;
+      if (sendQueue.empty()) return false;
+    }
+    element = sendQueue.front();
+    return true;
+  }
+
+  // Mark as processed and return time to schedule for next process.
+  // Return 0 when nothing to process.
+  // @param remove_from_queue indicates whether the elements should be removed from the queue.
+  unsigned long markProcessed(bool remove_from_queue) {
+    if (sendQueue.empty()) return 0;
+    if (remove_from_queue) {
+      sendQueue.pop_front();
+      attempt = 0;
+    } else {
+      ++attempt;
+    }
+    lastSend = millis();
+    return getNextScheduleTime();
+  }
+
+  unsigned long getNextScheduleTime() const {
+    if (sendQueue.empty()) return 0;
+    unsigned long nextTime = lastSend + minTimeBetweenMessages;
+    if (timePassedSince(nextTime) > 0) {
+      nextTime = millis();
+    }
+    if (nextTime == 0) nextTime = 1; // Just to make sure it will be executed
+    return nextTime;
+  }
+
+  size_t getQueueMemorySize() const {
+    size_t totalSize = 0;
+    for (auto it = sendQueue.begin(); it != sendQueue.end(); ++it) {
+      totalSize += it->getSize();
+    }
+    return totalSize;
+  }
+
+  std::list<T> sendQueue;
+  unsigned long lastSend;
+  unsigned int minTimeBetweenMessages;
+  byte max_queue_depth;
+  byte attempt;
+  byte max_retries;
+  bool delete_oldest;
+  bool must_check_reply;
+};
+
+ControllerDelayHandlerStruct<MQTT_queue_element> MQTTDelayHandler;
+
+
+// This macro defines the code needed to create the 'process_c##NNN##_delay_queue()'
+// function and all needed objects and forward declarations.
+// It is a macro to prevent common typo errors.
+// This function will perform the (re)scheduling and mark if it is processed (and can be removed)
+// The controller itself must implement the 'do_process_c004_delay_queue' function to actually
+// send the data.
+// Its return value must state whether it can be marked 'Processed'.
+// N.B. some controllers only can send one value per iteration, so a returned "false" can mean it
+//      was still successful. The controller should keep track of the last value sent
+//      in the element stored in the queue.
+#define DEFINE_Cxxx_DELAY_QUEUE_MACRO(NNN, M) \
+                ControllerDelayHandlerStruct<C##NNN##_queue_element> C##NNN##_DelayHandler; \
+                bool do_process_c##NNN##_delay_queue(int controller_number, const C##NNN##_queue_element& element, ControllerSettingsStruct& ControllerSettings); \
+                void process_c##NNN##_delay_queue() { \
+                  C##NNN##_queue_element element; \
+                  if (!C##NNN##_DelayHandler.getNext(element)) return; \
+                  ControllerSettingsStruct ControllerSettings; \
+                  LoadControllerSettings(element.controller_idx, ControllerSettings); \
+                  C##NNN##_DelayHandler.configureControllerSettings(ControllerSettings); \
+                  if (!WiFiConnected(100)) { \
+                    scheduleNextDelayQueue(TIMER_C##NNN##_DELAY_QUEUE, C##NNN##_DelayHandler.getNextScheduleTime()); \
+                    return; \
+                  } \
+                  START_TIMER; \
+                  C##NNN##_DelayHandler.markProcessed(do_process_c##NNN##_delay_queue(M, element, ControllerSettings)); \
+                  STOP_TIMER(C##NNN##_DELAY_QUEUE); \
+                  scheduleNextDelayQueue(TIMER_C##NNN##_DELAY_QUEUE, C##NNN##_DelayHandler.getNextScheduleTime()); \
+                }
+
+// Define the function wrappers to handle the calling to Cxxx_DelayHandler etc.
+// If someone knows how to add leading zeros in macros, please be my guest :)
+#ifdef USES_C001
+  DEFINE_Cxxx_DELAY_QUEUE_MACRO(001, 1)
+#endif
+#ifdef USES_C003
+  DEFINE_Cxxx_DELAY_QUEUE_MACRO(003, 3)
+#endif
+#ifdef USES_C004
+  DEFINE_Cxxx_DELAY_QUEUE_MACRO(004, 4)
+#endif
+#ifdef USES_C007
+  DEFINE_Cxxx_DELAY_QUEUE_MACRO(007, 7)
+#endif
+#ifdef USES_C008
+  DEFINE_Cxxx_DELAY_QUEUE_MACRO(008, 8)
+#endif
+#ifdef USES_C009
+  DEFINE_Cxxx_DELAY_QUEUE_MACRO(009, 9)
+#endif
+#ifdef USES_C010
+  DEFINE_Cxxx_DELAY_QUEUE_MACRO(010, 10)
+#endif
+#ifdef USES_C011
+  DEFINE_Cxxx_DELAY_QUEUE_MACRO(011, 11)
+#endif
+#ifdef USES_C012
+  DEFINE_Cxxx_DELAY_QUEUE_MACRO(012, 12)
+#endif
+/*
+#ifdef USES_C013
+  DEFINE_Cxxx_DELAY_QUEUE_MACRO(013, 13)
+#endif
+*/
+// When extending this, also extend in Scheduler.ino:
+// void process_interval_timer(unsigned long id, unsigned long lasttimer)
+
+
+/*********************************************************************************************\
+ * Helper functions used in a number of controllers
+\*********************************************************************************************/
+bool safeReadStringUntil(Stream &input, String &str, char terminator, unsigned int maxSize = 1024, unsigned int timeout = 1000)
+{
+	int c;
+  const unsigned long start = millis();
+	const unsigned long timer = start + timeout;
+  unsigned long backgroundtasks_timer = start + 10;
+	str = "";
+
+	do {
+		//read character
+		c = input.read();
+		if (c >= 0) {
+			//found terminator, we're ok
+			if (c == terminator) {
+				return(true);
+			}
+			//found character, add to string
+			else{
+				str += char(c);
+				//string at max size?
+				if (str.length() >= maxSize) {
+					addLog(LOG_LEVEL_ERROR, F("Not enough bufferspace to read all input data!"));
+					return(false);
+				}
+			}
+		}
+    // We must run the backgroundtasks every now and then.
+    if (timeOutReached(backgroundtasks_timer)) {
+      backgroundtasks_timer += 10;
+      backgroundtasks();
+    } else {
+      yield();
+    }
+	} while (!timeOutReached(timer));
+
+	addLog(LOG_LEVEL_ERROR, F("Timeout while reading input data!"));
+	return(false);
+}
+
+bool valid_controller_number(int controller_number) {
+  if (controller_number < 0) return false;
+  return true;
+//  return getProtocolIndex(controller_number) <= protocolCount;
+}
+
+String get_formatted_Controller_number(int controller_number) {
+  if (!valid_controller_number(controller_number)) {
+    return F("C---");
+  }
+  String result = F("C");
+  if (controller_number < 100) result += '0';
+  if (controller_number < 10) result += '0';
+  result += controller_number;
+  return result;
+}
+
+String get_auth_header(int controller_index) {
+  String authHeader = "";
+  if (controller_index < CONTROLLER_MAX) {
+    if ((SecuritySettings.ControllerUser[controller_index][0] != 0) &&
+        (SecuritySettings.ControllerPassword[controller_index][0] != 0))
+    {
+      base64 encoder;
+      String auth = SecuritySettings.ControllerUser[controller_index];
+      auth += ":";
+      auth += SecuritySettings.ControllerPassword[controller_index];
+      authHeader = F("Authorization: Basic ");
+      authHeader += encoder.encode(auth);
+      authHeader += F(" \r\n");
+    }
+  } else {
+    addLog(LOG_LEVEL_ERROR, F("Invalid controller index"));
+  }
+  return authHeader;
+}
+
+String get_user_agent_request_header_field() {
+  static unsigned int agent_size = 20;
+  String request;
+  request.reserve(agent_size);
+  request = F("User-Agent: ");
+  request += F("ESP Easy/");
+  request += BUILD;
+  request += '/';
+  request += String(CRCValues.compileDate);
+  request += ' ';
+  request += String(CRCValues.compileTime);
+  request += F("\r\n");
+  agent_size = request.length();
+  return request;
+}
+
+String do_create_http_request(
+    const String& hostportString,
+    const String& method, const String& uri,
+    const String& auth_header, const String& additional_options,
+    int content_length) {
+  int estimated_size = hostportString.length() + method.length()
+                       + uri.length() + auth_header.length()
+                       + additional_options.length()
+                       + 42;
+  if (content_length >= 0) estimated_size += 25;
+  String request;
+  request.reserve(estimated_size);
+  request += method;
+  request += ' ';
+  if (!uri.startsWith("/")) request += '/';
+  request += uri;
+  request += F(" HTTP/1.1\r\n");
+  if (content_length >= 0) {
+    request += F("Content-Length: ");
+    request += content_length;
+    request += F("\r\n");
+  }
+  request += F("Host: ");
+  request += hostportString;
+  request += F("\r\n");
+  request += auth_header;
+  request += additional_options;
+  request += get_user_agent_request_header_field();
+  request += F("Connection: close\r\n");
+  request += F("\r\n");
+  addLog(LOG_LEVEL_DEBUG, request);
+  return request;
+}
+
+String do_create_http_request(
+    const String& hostportString,
+    const String& method, const String& uri) {
+  return do_create_http_request(hostportString, method, uri,
+    "", // auth_header
+    "", // additional_options
+    -1  // content_length
+  );
+}
+
+String do_create_http_request(
+    int controller_number, ControllerSettingsStruct& ControllerSettings,
+    const String& method, const String& uri,
+    int content_length) {
+  const bool defaultport = ControllerSettings.Port == 0 || ControllerSettings.Port == 80;
+  return do_create_http_request(
+    defaultport ? ControllerSettings.getHost() : ControllerSettings.getHostPortString(),
+    method,
+    uri,
+    "", // auth_header
+    "", // additional_options
+    content_length);
+}
+
+String create_http_request_auth(
+    int controller_number, int controller_index, ControllerSettingsStruct& ControllerSettings,
+    const String& method, const String& uri,
+    int content_length) {
+  const bool defaultport = ControllerSettings.Port == 0 || ControllerSettings.Port == 80;
+  return do_create_http_request(
+    defaultport ? ControllerSettings.getHost() : ControllerSettings.getHostPortString(),
+    method,
+    uri,
+    get_auth_header(controller_index),
+    "", // additional_options
+    content_length);
+}
+
+String create_http_get_request(int controller_number, ControllerSettingsStruct& ControllerSettings,
+    const String& uri) {
+  return do_create_http_request(controller_number, ControllerSettings, F("GET"), uri, -1);
+}
+
+String create_http_request_auth(int controller_number, int controller_index, ControllerSettingsStruct& ControllerSettings,
+    const String& method, const String& uri) {
+  return create_http_request_auth(controller_number, controller_index, ControllerSettings, method, uri, -1);
+}
+
+void log_connecting_to(const String& prefix, int controller_number, ControllerSettingsStruct& ControllerSettings) {
+  if (loglevelActiveFor(LOG_LEVEL_DEBUG)) {
+    String log = prefix;
+    log += get_formatted_Controller_number(controller_number);
+    log += F(" connecting to ");
+    log += ControllerSettings.getHostPortString();
+    addLog(LOG_LEVEL_DEBUG, log);
+  }
+}
+
+void log_connecting_fail(const String& prefix, int controller_number, ControllerSettingsStruct& ControllerSettings) {
+  if (loglevelActiveFor(LOG_LEVEL_ERROR)) {
+    String log = prefix;
+    log += get_formatted_Controller_number(controller_number);
+    log += F(" connection failed");
+    addLog(LOG_LEVEL_ERROR, log);
+  }
+}
+
+bool count_connection_results(bool success, const String& prefix, int controller_number, ControllerSettingsStruct& ControllerSettings) {
+  if (!success)
+  {
+    connectionFailures++;
+    log_connecting_fail(prefix, controller_number, ControllerSettings);
+    return false;
+  }
+  statusLED(true);
+  if (connectionFailures)
+    connectionFailures--;
+  return true;
+}
+
+bool try_connect_host(int controller_number, WiFiUDP& client, ControllerSettingsStruct& ControllerSettings) {
+  client.setTimeout(ControllerSettings.ClientTimeout);
+  log_connecting_to(F("UDP  : "), controller_number, ControllerSettings);
+  bool success = ControllerSettings.beginPacket(client) != 0;
+  return count_connection_results(
+      success,
+      F("UDP  : "), controller_number, ControllerSettings);
+}
+
+bool try_connect_host(int controller_number, WiFiClient& client, ControllerSettingsStruct& ControllerSettings) {
+  // Use WiFiClient class to create TCP connections
+  client.setTimeout(ControllerSettings.ClientTimeout);
+  log_connecting_to(F("HTTP : "), controller_number, ControllerSettings);
+  bool success = ControllerSettings.connectToHost(client);
+  return count_connection_results(
+      success,
+      F("HTTP : "), controller_number, ControllerSettings);
+}
+
+// Use "client.available() || client.connected()" to read all lines from slow servers.
+// See: https://github.com/esp8266/Arduino/pull/5113
+//      https://github.com/esp8266/Arduino/pull/1829
+bool client_available(WiFiClient& client) {
+  return client.available() || client.connected();
+}
+
+bool send_via_http(const String& logIdentifier, WiFiClient& client, const String& postStr, bool must_check_reply) {
+  bool success = !must_check_reply;
+  // This will send the request to the server
+  client.print(postStr);
+
+  if (must_check_reply) {
+    unsigned long timer = millis() + 200;
+    while (!client.available()) {
+      if (timeOutReached(timer)) return false;
+      delay(1);
+    }
+
+    // Read all the lines of the reply from server and print them to Serial
+    while (client_available(client) && !success) {
+      //   String line = client.readStringUntil('\n');
+      String line;
+      safeReadStringUntil(client, line, '\n');
+
+      if (loglevelActiveFor(LOG_LEVEL_DEBUG_MORE)) {
+        if (line.length() > 80) {
+          addLog(LOG_LEVEL_DEBUG_MORE, line.substring(0, 80));
+        } else {
+          addLog(LOG_LEVEL_DEBUG_MORE, line);
+        }
+      }
+      if (line.startsWith(F("HTTP/1.1 2")))
+      {
+        success = true;
+        if (loglevelActiveFor(LOG_LEVEL_DEBUG)) {
+          String log = F("HTTP : ");
+          log += logIdentifier;
+          log += F(" Success! ");
+          log += line;
+          addLog(LOG_LEVEL_DEBUG, log);
+        }
+      } else if (line.startsWith(F("HTTP/1.1 4"))) {
+        if (loglevelActiveFor(LOG_LEVEL_ERROR)) {
+          String log = F("HTTP : ");
+          log += logIdentifier;
+          log += F(" Error: ");
+          log += line;
+          addLog(LOG_LEVEL_ERROR, log);
+        }
+        addLog(LOG_LEVEL_DEBUG_MORE, postStr);
+      }
+      yield();
+    }
+  }
+  if (loglevelActiveFor(LOG_LEVEL_DEBUG)) {
+    String log = F("HTTP : ");
+    log += logIdentifier;
+    log += F(" closing connection");
+    addLog(LOG_LEVEL_DEBUG, log);
+  }
+
+  client.flush();
+  client.stop();
+  return success;
+}
+
+bool send_via_http(int controller_number, WiFiClient& client, const String& postStr, bool must_check_reply) {
+  return send_via_http(get_formatted_Controller_number(controller_number), client, postStr, must_check_reply);
+}
+
+
+#endif // CPLUGIN_HELPER_H

src/_N001_Email.ino

@@ -73,6 +73,7 @@ boolean NPlugin_001_send(const NotificationSettingsStruct& notificationsettings,
 
 	// Use WiFiClient class to create TCP connections
 	WiFiClient client;
+	client.setTimeout(CONTROLLER_CLIENTTIMEOUT_DFLT);
 	String aHost = notificationsettings.Server;
 	addLog(LOG_LEVEL_DEBUG, String(F("EMAIL: Connecting to ")) + aHost + notificationsettings.Port);
 	if (client.connect(aHost.c_str(), notificationsettings.Port) != 1) {
@@ -89,6 +90,24 @@ boolean NPlugin_001_send(const NotificationSettingsStruct& notificationsettings,
 			"X-Mailer: EspEasy v$espeasyversion\r\n\r\n"
 			);
 
+    String email_address = notificationsettings.Sender;
+		int pos_less = email_address.indexOf('<');
+		if (pos_less == -1) {
+			// No email address markup
+			mailheader.replace(String(F("$nodename")), Settings.Name);
+			mailheader.replace(String(F("$emailfrom")), notificationsettings.Sender);
+		} else {
+			String senderName = email_address.substring(0, pos_less);
+			senderName.replace("\"", ""); // Remove quotes
+			String address = email_address.substring(pos_less + 1);
+			address.replace("<", "");
+			address.replace(">", "");
+			address.trim();
+			senderName.trim();
+			mailheader.replace(String(F("$nodename")), senderName);
+			mailheader.replace(String(F("$emailfrom")), address);
+		}
+
 		mailheader.replace(String(F("$nodename")), Settings.Name);
 		mailheader.replace(String(F("$emailfrom")), notificationsettings.Sender);
 		mailheader.replace(String(F("$ato")), notificationsettings.Receiver);

src/_P001_Switch.ino

@@ -257,6 +257,22 @@ boolean Plugin_001(byte function, struct EventStruct *event, String& string)
           }
         }
 
+        if (command == F("gpiotoggle"))
+        {
+          success = true;
+          byte mode;
+          uint16_t currentState;
+
+          getPinState(PLUGIN_ID_001, event->Par1, &mode, &currentState);
+          if (mode == PIN_MODE_OUTPUT || mode == PIN_MODE_UNDEFINED) { //toggle only output pins
+            digitalWrite(event->Par1, !currentState);
+            setPinState(PLUGIN_ID_001, event->Par1, PIN_MODE_OUTPUT, !currentState);
+            log = String(F("SW   : Toggle GPIO ")) + String(event->Par1) + String(F(" Set to ")) + String(!currentState);
+            addLog(LOG_LEVEL_INFO, log);
+            SendStatus(event->Source, getPinStateJSON(SEARCH_PIN_STATE, PLUGIN_ID_001, event->Par1, log, 0));
+          }
+        }
+
         if (command == F("pwm"))
         {
           success = true;

src/_P004_Dallas.ino

@@ -90,7 +90,7 @@ boolean Plugin_004(byte function, struct EventStruct * event, String& string)
                   resolutionChoice = Plugin_004_DS_getResolution(savedAddress);
               else
                   resolutionChoice = 9;
-              String resultsOptions[4] = { "9", "10", "11", "12" };
+              String resultsOptions[4] = { F("9"), F("10"), F("11"), F("12") };
               int resultsOptionValues[4] = { 9, 10, 11, 12 };
               addFormSelector(F("Device Resolution"), F("plugin_004_res"), 4, resultsOptions, resultsOptionValues, resolutionChoice);
               addHtml(F(" Bit"));
@@ -105,7 +105,6 @@ boolean Plugin_004(byte function, struct EventStruct * event, String& string)
 
             // save the address for selected device and store into extra tasksettings
             Plugin_004_DallasPin = Settings.TaskDevicePin1[event->TaskIndex];
-            // byte devCount =
             if (Plugin_004_DallasPin != -1){
               Plugin_004_DS_scan(getFormItemInt(F("plugin_004_dev")), addr);
               for (byte x = 0; x < 8; x++)
@@ -124,11 +123,12 @@ boolean Plugin_004(byte function, struct EventStruct * event, String& string)
             {
                 if (x != 0)
                     string += "-";
-                // string += String(ExtraTaskSettings.TaskDevicePluginConfigLong[x], HEX);
+                string += String(ExtraTaskSettings.TaskDevicePluginConfigLong[x], HEX);
             }
             success = true;
             break;
         }
+
         case PLUGIN_INIT:
         {
             Plugin_004_DallasPin = Settings.TaskDevicePin1[event->TaskIndex];
@@ -169,7 +169,7 @@ boolean Plugin_004(byte function, struct EventStruct * event, String& string)
                 for (byte x = 0; x < 8; x++)
                 {
                     if (x != 0)
-                        log += "-";
+                        log += '-';
                     log += String(ExtraTaskSettings.TaskDevicePluginConfigLong[x], HEX);
                 }
 

src/_P005_DHT.ino

@@ -9,6 +9,19 @@
 #define PLUGIN_VALUENAME1_005 "Temperature"
 #define PLUGIN_VALUENAME2_005 "Humidity"
 
+#define P005_DHT11    11
+#define P005_DHT12    12
+#define P005_DHT22    22
+#define P005_AM2301   23
+#define P005_SI7021   70
+
+#define P005_error_no_reading          1
+#define P005_error_protocol_timeout    2
+#define P005_error_checksum_error      3
+#define P005_error_invalid_NAN_reading 4
+#define P005_info_temperature          5
+#define P005_info_humidity             6
+
 uint8_t Plugin_005_DHT_Pin;
 
 boolean Plugin_005(byte function, struct EventStruct *event, String& string)
@@ -49,7 +62,7 @@ boolean Plugin_005(byte function, struct EventStruct *event, String& string)
     case PLUGIN_WEBFORM_LOAD:
       {
         const String options[] = { F("DHT 11"), F("DHT 22"), F("DHT 12"), F("Sonoff am2301"), F("Sonoff si7021") };
-        int indices[] = { 11, 22, 12, 23, 70 };
+        int indices[] = { P005_DHT11, P005_DHT22, P005_DHT12, P005_AM2301, P005_SI7021 };
 
         addFormSelector(F("DHT Type"), F("plugin_005_dhttype"), 5, options, indices, Settings.TaskDevicePluginConfig[event->TaskIndex][0] );
 
@@ -67,92 +80,7 @@ boolean Plugin_005(byte function, struct EventStruct *event, String& string)
 
     case PLUGIN_READ:
       {
-        byte dht_dat[5];
-        byte i;
-        boolean error = false;
-
-        byte Par3 = Settings.TaskDevicePluginConfig[event->TaskIndex][0];
-        Plugin_005_DHT_Pin = Settings.TaskDevicePin1[event->TaskIndex];
-
-        pinMode(Plugin_005_DHT_Pin, OUTPUT);
-        digitalWrite(Plugin_005_DHT_Pin, LOW);              // Pull low
-        if(Par3 == 11 || Par3 == 22 || Par3 == 12)  delay(18);
-        else if (Par3 == 23 )         delayMicroseconds(900);
-        else if (Par3 == 70 )         delayMicroseconds(500);
-        pinMode(Plugin_005_DHT_Pin, INPUT);                 // change pin to input
-        delayMicroseconds(50);
-
-        error = waitState(0);
-        if(error)
-        {   logError(event, F("DHT  : no Reading !"));
-            break;
-        }
-        error = waitState(1);
-        if(error)
-        {   logError(event, F("DHT  : no Reading !"));
-            break;
-        }
-        noInterrupts();
-        error = waitState(0);
-        if(error)
-        {   logError(event, F("DHT  : no Reading !"));
-            break;
-        }
-        for (i = 0; i < 5; i++)
-        {
-            byte data = Plugin_005_read_dht_dat();
-            if(data == -1)
-            {   logError(event, F("DHT  : protocol timeout!"));
-                break;
-            }
-            dht_dat[i] = data;
-        }
-        interrupts();
-
-              // Checksum calculation is a Rollover Checksum by design!
-        byte dht_check_sum = (dht_dat[0] + dht_dat[1] + dht_dat[2] + dht_dat[3]) & 0xFF; // check check_sum
-        if (dht_dat[4] != dht_check_sum)
-        {
-            logError(event, F("DHT  : checksum error!"));
-            break;
-        }
-
-        float temperature = NAN;
-        float humidity = NAN;
-        if (Par3 == 11)
-        {
-          temperature = float(dht_dat[2]); // Temperature
-          humidity = float(dht_dat[0]); // Humidity
-        }
-        else if (Par3 == 12)
-        {
-            temperature = float(dht_dat[2]*10 + (dht_dat[3] & 0x7f)) / 10.0; // Temperature
-            if (dht_dat[3] & 0x80) { temperature = -temperature; } // Negative temperature
-            humidity = float(dht_dat[0]*10+dht_dat[1]) / 10.0; // Humidity
-        }
-        else if (Par3 == 22 || Par3 == 23 || Par3 == 70)
-        {
-          if (dht_dat[2] & 0x80) // negative temperature
-            temperature = -0.1 * word(dht_dat[2] & 0x7F, dht_dat[3]);
-          else
-            temperature = 0.1 * word(dht_dat[2], dht_dat[3]);
-          humidity = 0.1 * word(dht_dat[0], dht_dat[1]); // Humidity
-        }
-
-        if (temperature == NAN || humidity == NAN)
-        {     logError(event, F("DHT  : invalid NAN reading !"));
-              break;
-        }
-
-        UserVar[event->BaseVarIndex] = temperature;
-        UserVar[event->BaseVarIndex + 1] = humidity;
-        String log = F("DHT  : Temperature: ");
-        log += UserVar[event->BaseVarIndex];
-        addLog(LOG_LEVEL_INFO, log);
-        log = F("DHT  : Humidity: ");
-        log += UserVar[event->BaseVarIndex + 1];
-        addLog(LOG_LEVEL_INFO, log);
-        success = true;
+        success = P005_do_plugin_read(event);
         break;
       }
   }
@@ -163,28 +91,144 @@ boolean Plugin_005(byte function, struct EventStruct *event, String& string)
 /*********************************************************************************************\
 * DHT sub to log an error
 \*********************************************************************************************/
-void logError(struct EventStruct *event, String text)
+void P005_log(struct EventStruct *event, int logNr)
 {
+  bool isError = true;
+  String text = F("DHT  : ");
+  switch (logNr) {
+    case P005_error_no_reading:          text += F("No Reading"); break;
+    case P005_error_protocol_timeout:    text += F("Protocol Timeout"); break;
+    case P005_error_checksum_error:      text += F("Checksum Error"); break;
+    case P005_error_invalid_NAN_reading: text += F("Invalid NAN reading"); break;
+    case P005_info_temperature:
+      text += F("Temperature: ");
+      text += UserVar[event->BaseVarIndex];
+      isError = false;
+      break;
+    case P005_info_humidity:
+      text += F("Humidity: ");
+      text += UserVar[event->BaseVarIndex + 1];
+      isError = false;
+      break;
+  }
   addLog(LOG_LEVEL_INFO, text);
-  UserVar[event->BaseVarIndex] = NAN;
-  UserVar[event->BaseVarIndex + 1] = NAN;
+  if (isError) {
+    UserVar[event->BaseVarIndex] = NAN;
+    UserVar[event->BaseVarIndex + 1] = NAN;
+  }
 }
 
 /*********************************************************************************************\
-* DHT sub to wait until a pin is in a certiin state
+* DHT sub to wait until a pin is in a certain state
 \*********************************************************************************************/
-boolean waitState(int state)
+boolean P005_waitState(int state)
 {
-  byte counter = 0;
-  while (( digitalRead(Plugin_005_DHT_Pin) != state) && (counter < 100))
+  unsigned long timeout = micros() + 100;
+  while (digitalRead(Plugin_005_DHT_Pin) != state)
   {
+    if (usecTimeOutReached(timeout)) return false;
     delayMicroseconds(1);
-    counter++;
   }
-  if( counter < 100) return false;
   return true;
 }
 
+/*********************************************************************************************\
+* Perform the actual reading + interpreting of data.
+\*********************************************************************************************/
+bool P005_do_plugin_read(struct EventStruct *event) {
+  byte i;
+
+  byte Par3 = Settings.TaskDevicePluginConfig[event->TaskIndex][0];
+  Plugin_005_DHT_Pin = Settings.TaskDevicePin1[event->TaskIndex];
+
+  pinMode(Plugin_005_DHT_Pin, OUTPUT);
+  digitalWrite(Plugin_005_DHT_Pin, LOW);              // Pull low
+  switch (Par3) {
+    case P005_DHT11:
+    case P005_DHT22:
+    case P005_DHT12:  delay(18); break;  // FIXME TD-er: Must this be so long?
+    case P005_AM2301: delayMicroseconds(900); break;
+    case P005_SI7021: delayMicroseconds(500); break;
+  }
+  switch (Par3) {
+    case P005_DHT11:
+    case P005_DHT22:
+    case P005_DHT12:
+    case P005_AM2301:
+      pinMode(Plugin_005_DHT_Pin, INPUT);
+      delayMicroseconds(50);
+      break;
+    case P005_SI7021:
+      // See: https://github.com/letscontrolit/ESPEasy/issues/1798
+      digitalWrite(Plugin_005_DHT_Pin, HIGH);
+      delayMicroseconds(20);
+      pinMode(Plugin_005_DHT_Pin, INPUT);
+      break;
+  }
+  if(!P005_waitState(0)) {P005_log(event, P005_error_no_reading); return false; }
+  if(!P005_waitState(1)) {P005_log(event, P005_error_no_reading); return false; }
+  noInterrupts();
+  if(!P005_waitState(0)) {P005_log(event, P005_error_no_reading); return false; }
+  bool readingAborted = false;
+  byte dht_dat[5];
+  for (i = 0; i < 5 && !readingAborted; i++)
+  {
+      byte data = Plugin_005_read_dht_dat();
+      if(data == -1)
+      {   P005_log(event, P005_error_protocol_timeout);
+          readingAborted = true;
+      }
+      dht_dat[i] = data;
+  }
+  interrupts();
+  if (readingAborted)
+    return false;
+
+        // Checksum calculation is a Rollover Checksum by design!
+  byte dht_check_sum = (dht_dat[0] + dht_dat[1] + dht_dat[2] + dht_dat[3]) & 0xFF; // check check_sum
+  if (dht_dat[4] != dht_check_sum)
+  {
+      P005_log(event, P005_error_checksum_error);
+      return false;
+  }
+
+  float temperature = NAN;
+  float humidity = NAN;
+  switch (Par3) {
+    case P005_DHT11:
+      temperature = float(dht_dat[2]); // Temperature
+      humidity = float(dht_dat[0]); // Humidity
+      break;
+    case P005_DHT12:
+      temperature = float(dht_dat[2]*10 + (dht_dat[3] & 0x7f)) / 10.0; // Temperature
+      if (dht_dat[3] & 0x80) { temperature = -temperature; } // Negative temperature
+      humidity = float(dht_dat[0]*10+dht_dat[1]) / 10.0; // Humidity
+      break;
+    case P005_DHT22:
+    case P005_AM2301:
+    case P005_SI7021:
+      if (dht_dat[2] & 0x80) // negative temperature
+        temperature = -0.1 * word(dht_dat[2] & 0x7F, dht_dat[3]);
+      else
+        temperature = 0.1 * word(dht_dat[2], dht_dat[3]);
+      humidity = 0.1 * word(dht_dat[0], dht_dat[1]); // Humidity
+      break;
+  }
+
+  if (temperature == NAN || humidity == NAN)
+  {     P005_log(event, P005_error_invalid_NAN_reading);
+        return false;
+  }
+
+  UserVar[event->BaseVarIndex] = temperature;
+  UserVar[event->BaseVarIndex + 1] = humidity;
+  P005_log(event, P005_info_temperature);
+  P005_log(event, P005_info_humidity);
+  return true;
+}
+
+
+
 /*********************************************************************************************\
 * DHT sub to get an 8 bit value from the receiving bitstream
 \*********************************************************************************************/
@@ -192,31 +236,13 @@ int Plugin_005_read_dht_dat(void)
 {
   byte i = 0;
   byte result = 0;
-  byte counter = 0;
   for (i = 0; i < 8; i++)
   {
-    while ((!digitalRead(Plugin_005_DHT_Pin)) && (counter < 100))
-    {
-      delayMicroseconds(1);
-      counter++;
-    }
-    if (counter >= 100)
-    {
-      return -1;
-    }
+    if (!P005_waitState(1))  return -1;
     delayMicroseconds(35); // was 30
     if (digitalRead(Plugin_005_DHT_Pin))
       result |= (1 << (7 - i));
-    counter = 0;
-    while ((digitalRead(Plugin_005_DHT_Pin)) && (counter < 100))
-    {
-      delayMicroseconds(1);
-      counter++;
-    }
-    if (counter >= 100)
-    {
-      return -1;
-    }
+    if (!P005_waitState(0))  return -1;
   }
   return result;
 }

src/_P009_MCP.ino

@@ -20,7 +20,7 @@ boolean Plugin_009(byte function, struct EventStruct *event, String& string)
       {
         Device[++deviceCount].Number = PLUGIN_ID_009;
         Device[deviceCount].Type = DEVICE_TYPE_I2C;
-        Device[deviceCount].VType = SENSOR_TYPE_SINGLE;
+        Device[deviceCount].VType = SENSOR_TYPE_SWITCH;
         Device[deviceCount].Ports = 16;
         Device[deviceCount].PullUpOption = false;
         Device[deviceCount].InverseLogicOption = false;
@@ -124,6 +124,22 @@ boolean Plugin_009(byte function, struct EventStruct *event, String& string)
           SendStatus(event->Source, getPinStateJSON(SEARCH_PIN_STATE, PLUGIN_ID_009, event->Par1, log, 0));
         }
 
+        if (command == F("mcpgpiotoggle"))
+        {
+          success = true;
+          byte mode;
+          uint16_t currentState;
+
+          getPinState(PLUGIN_ID_009, event->Par1, &mode, &currentState);
+          if (mode == PIN_MODE_OUTPUT || mode == PIN_MODE_UNDEFINED) {
+            setPinState(PLUGIN_ID_009, event->Par1, PIN_MODE_OUTPUT, !currentState);
+            Plugin_009_Write(event->Par1, !currentState);
+            log = String(F("MCP  : Toggle GPIO ")) + String(event->Par1) + String(F(" Set to ")) + String(!currentState);
+            addLog(LOG_LEVEL_INFO, log);
+            SendStatus(event->Source, getPinStateJSON(SEARCH_PIN_STATE, PLUGIN_ID_009, event->Par1, log, 0));
+          }
+        }
+
         if (command == F("mcppulse"))
         {
           success = true;

src/_P013_HCSR04.ino

@@ -12,27 +12,21 @@
 #include <map>
 #include <NewPingESP8266.h>
 
-struct P_013_sensordef {
-  P_013_sensordef() : sonar(NULL) {}
-
-  P_013_sensordef(byte TRIG_Pin, byte IRQ_Pin, int16_t max_cm_distance) : sonar(NULL) {
-    sonar = new NewPingESP8266(TRIG_Pin, IRQ_Pin, max_cm_distance);
-  }
-
-  ~P_013_sensordef() {
-    if (sonar != NULL) {
-      delete sonar;
-      sonar = NULL;
-    }
-  }
-
-  NewPingESP8266 *sonar;
-};
-
-std::map<unsigned int, P_013_sensordef> P_013_sensordefs;
+// PlugIn specific defines
+// operatingMode
+#define OPMODE_VALUE        (0)
+#define OPMODE_STATE        (1)
 
+// measuringUnit
+#define UNIT_CM             (0)
+#define UNIT_INCH           (1)
 
+// filterType
+#define FILTER_NONE         (0)
+#define FILTER_MEDIAN       (1)
 
+// map of sensors
+std::map<unsigned int, std::shared_ptr<NewPingESP8266> > P_013_sensordefs;
 
 boolean Plugin_013(byte function, struct EventStruct *event, String& string)
 {
@@ -54,6 +48,7 @@ boolean Plugin_013(byte function, struct EventStruct *event, String& string)
         Device[deviceCount].SendDataOption = true;
         Device[deviceCount].TimerOption = true;
         Device[deviceCount].GlobalSyncOption = true;
+
         break;
       }
 
@@ -72,19 +67,51 @@ boolean Plugin_013(byte function, struct EventStruct *event, String& string)
 
     case PLUGIN_WEBFORM_LOAD:
       {
-        byte choice = Settings.TaskDevicePluginConfig[event->TaskIndex][0];
-        String options[2];
-        options[0] = F("Value");
-        options[1] = F("State");
-        int optionValues[2] = { 1, 2 };
-        addFormSelector(F("Mode"), F("plugin_013_mode"), 2, options, optionValues, choice);
+        int16_t operatingMode = Settings.TaskDevicePluginConfig[event->TaskIndex][0];
+        int16_t threshold = Settings.TaskDevicePluginConfig[event->TaskIndex][1];
+        int16_t max_distance = Settings.TaskDevicePluginConfig[event->TaskIndex][2];
+        int16_t measuringUnit = Settings.TaskDevicePluginConfig[event->TaskIndex][3];
+        int16_t filterType = Settings.TaskDevicePluginConfig[event->TaskIndex][4];
+        int16_t filterSize = Settings.TaskDevicePluginConfig[event->TaskIndex][5];
 
-        if (Settings.TaskDevicePluginConfig[event->TaskIndex][0] == 2)
-        {
-        	addFormNumericBox(F("Threshold"), F("plugin_013_threshold"), Settings.TaskDevicePluginConfig[event->TaskIndex][1]);
+        // default filtersize = 5
+        if (filterSize == 0) {
+          filterSize = 5;
+          Settings.TaskDevicePluginConfig[event->TaskIndex][5] = filterSize;
         }
-        addFormNumericBox(F("Max Distance"), F("plugin_013_max_distance"), Settings.TaskDevicePluginConfig[event->TaskIndex][2], 0, 500);
-        addUnit(F("cm"));
+
+
+        String strUnit = (measuringUnit == UNIT_CM) ? F("cm") : F("inch");
+
+        String optionsOpMode[2];
+        int optionValuesOpMode[2] = { 0, 1 };
+        optionsOpMode[0] = F("Value");
+        optionsOpMode[1] = F("State");
+        addFormSelector(F("Mode"), F("plugin_013_mode"), 2, optionsOpMode, optionValuesOpMode, operatingMode);
+
+        if (operatingMode == OPMODE_STATE)
+        {
+        	addFormNumericBox(F("Threshold"), F("plugin_013_threshold"), threshold);
+          addUnit(strUnit);
+        }
+        addFormNumericBox(F("Max Distance"), F("plugin_013_max_distance"), max_distance, 0, 500);
+        addUnit(strUnit);
+
+        String optionsUnit[2];
+        int optionValuesUnit[2] = { 0, 1 };
+        optionsUnit[0] = F("Metric");
+        optionsUnit[1] = F("Imperial");
+        addFormSelector(F("Unit"), F("plugin_013_Unit"), 2, optionsUnit, optionValuesUnit, measuringUnit);
+
+        String optionsFilter[2];
+        int optionValuesFilter[2] = { 0, 1 };
+        optionsFilter[0] = F("None");
+        optionsFilter[1] = F("Median");
+        addFormSelector(F("Filter"), F("plugin_013_FilterType"), 2, optionsFilter, optionValuesFilter, filterType);
+
+        // enable filtersize option if filter is used,
+        if (filterType != FILTER_NONE)
+        	addFormNumericBox(F("Filter size"), F("plugin_013_FilterSize"), filterSize, 2, 20);
 
         success = true;
         break;
@@ -92,33 +119,60 @@ boolean Plugin_013(byte function, struct EventStruct *event, String& string)
 
     case PLUGIN_WEBFORM_SAVE:
       {
+        int16_t operatingMode = Settings.TaskDevicePluginConfig[event->TaskIndex][0];
+        int16_t filterType = Settings.TaskDevicePluginConfig[event->TaskIndex][4];
+
         Settings.TaskDevicePluginConfig[event->TaskIndex][0] = getFormItemInt(F("plugin_013_mode"));
-        if (Settings.TaskDevicePluginConfig[event->TaskIndex][0] == 2)
-        {
+        if (operatingMode == OPMODE_STATE)
           Settings.TaskDevicePluginConfig[event->TaskIndex][1] = getFormItemInt(F("plugin_013_threshold"));
-        }
         Settings.TaskDevicePluginConfig[event->TaskIndex][2] = getFormItemInt(F("plugin_013_max_distance"));
+
+        Settings.TaskDevicePluginConfig[event->TaskIndex][3] = getFormItemInt(F("plugin_013_Unit"));
+        Settings.TaskDevicePluginConfig[event->TaskIndex][4] = getFormItemInt(F("plugin_013_FilterType"));
+        if (filterType != FILTER_NONE)
+          Settings.TaskDevicePluginConfig[event->TaskIndex][5] = getFormItemInt(F("plugin_013_FilterSize"));
+
         success = true;
         break;
       }
 
     case PLUGIN_INIT:
       {
-        byte Plugin_013_TRIG_Pin = Settings.TaskDevicePin1[event->TaskIndex];
-        byte Plugin_013_IRQ_Pin = Settings.TaskDevicePin2[event->TaskIndex];
-        int16_t max_cm_distance = Settings.TaskDevicePluginConfig[event->TaskIndex][2];
+        int16_t max_distance = Settings.TaskDevicePluginConfig[event->TaskIndex][2];
+        int16_t measuringUnit = Settings.TaskDevicePluginConfig[event->TaskIndex][3];
+        int16_t filterType = Settings.TaskDevicePluginConfig[event->TaskIndex][4];
+        int16_t filterSize = Settings.TaskDevicePluginConfig[event->TaskIndex][5];
+
+        int8_t Plugin_013_TRIG_Pin = Settings.TaskDevicePin1[event->TaskIndex];
+        int8_t Plugin_013_IRQ_Pin = Settings.TaskDevicePin2[event->TaskIndex];
+        int16_t max_distance_cm = (measuringUnit == UNIT_CM) ? max_distance : (float)max_distance * 2.54f;
+
+        // create sensor instance and add to std::map
+        P_013_sensordefs.erase(event->TaskIndex);
         P_013_sensordefs[event->TaskIndex] =
-          P_013_sensordef(Plugin_013_TRIG_Pin, Plugin_013_IRQ_Pin, max_cm_distance);
+          std::shared_ptr<NewPingESP8266> (new NewPingESP8266(Plugin_013_TRIG_Pin, Plugin_013_IRQ_Pin, max_distance_cm));
+
         String log = F("ULTRASONIC : TaskNr: ");
         log += event->TaskIndex +1;
         log += F(" TrigPin: ");
         log += Plugin_013_TRIG_Pin;
         log += F(" IRQ_Pin: ");
         log += Plugin_013_IRQ_Pin;
-        log += F(" max dist cm: ");
-        log += max_cm_distance;
+        log += F(" max dist ");
+        log += (measuringUnit == UNIT_CM) ? F("[cm]: ") : F("[inch]: ");
+        log += max_distance;
         log += F(" max echo: ");
-        log += P_013_sensordefs[event->TaskIndex].sonar->getMaxEchoTime();
+        log += P_013_sensordefs[event->TaskIndex]->getMaxEchoTime();
+        log += F(" Filter: ");
+        if (filterType == FILTER_NONE)
+          log += F("none");
+        else
+          if (filterType == FILTER_MEDIAN) {
+            log += F("Median size: ");
+            log += filterSize;
+          }
+          else
+            log += F("invalid!");
         log += F(" nr_tasks: ");
         log += P_013_sensordefs.size();
         addLog(LOG_LEVEL_INFO, log);
@@ -136,7 +190,6 @@ boolean Plugin_013(byte function, struct EventStruct *event, String& string)
         log += F(" usec, ");
         addLog(LOG_LEVEL_INFO, log);
 
-
         success = true;
         break;
       }
@@ -149,39 +202,48 @@ boolean Plugin_013(byte function, struct EventStruct *event, String& string)
 
     case PLUGIN_READ: // If we select value mode, read and send the value based on global timer
       {
-        if (Settings.TaskDevicePluginConfig[event->TaskIndex][0] == 1)
+        int16_t operatingMode = Settings.TaskDevicePluginConfig[event->TaskIndex][0];
+        int16_t measuringUnit = Settings.TaskDevicePluginConfig[event->TaskIndex][3];
+
+        if (operatingMode == OPMODE_VALUE)
         {
           float value = Plugin_013_read(event->TaskIndex);
           String log = F("ULTRASONIC : TaskNr: ");
           log += event->TaskIndex +1;
           log += F(" Distance: ");
-          if (value > 0)
+          UserVar[event->BaseVarIndex] = value;
+          log += UserVar[event->BaseVarIndex];
+          log += (measuringUnit == UNIT_CM) ? F(" cm ") : F(" inch ");
+          if (value == NO_ECHO)
           {
-            UserVar[event->BaseVarIndex] = value;
-            log += UserVar[event->BaseVarIndex];
-            success = true;
+             log += F(" Error: ");
+             log += Plugin_013_getErrorStatusString(event->TaskIndex);
           }
-          else
-            log += F("No reading!");
 
-        addLog(LOG_LEVEL_INFO,log);
+          addLog(LOG_LEVEL_INFO,log);
         }
+        success = true;
         break;
       }
 
     case PLUGIN_TEN_PER_SECOND: // If we select state mode, do more frequent checks and send only state changes
       {
-        if (Settings.TaskDevicePluginConfig[event->TaskIndex][0] == 2)
+        int16_t operatingMode = Settings.TaskDevicePluginConfig[event->TaskIndex][0];
+        int16_t threshold = Settings.TaskDevicePluginConfig[event->TaskIndex][1];
+
+        if (operatingMode == OPMODE_STATE)
         {
           byte state = 0;
           float value = Plugin_013_read(event->TaskIndex);
-          if (value > 0)
+          if (value != NO_ECHO)
           {
-            if (value < Settings.TaskDevicePluginConfig[event->TaskIndex][1])
+            if (value < threshold)
               state = 1;
             if (state != switchstate[event->TaskIndex])
             {
-              String log = F("ULTRASONIC : State ");
+              String log = F("ULTRASONIC : TaskNr: ");
+              log += event->TaskIndex +1;
+              log += F(" state: ");
               log += state;
               addLog(LOG_LEVEL_INFO,log);
               switchstate[event->TaskIndex] = state;
@@ -190,6 +252,14 @@ boolean Plugin_013(byte function, struct EventStruct *event, String& string)
               sendData(event);
             }
           }
+          else {
+            String log = F("ULTRASONIC : TaskNr: ");
+            log += event->TaskIndex +1;
+            log += F(" Error: ");
+            log += Plugin_013_getErrorStatusString(event->TaskIndex);
+            addLog(LOG_LEVEL_INFO,log);
+          }
+
         }
         success = true;
         break;
@@ -202,11 +272,82 @@ boolean Plugin_013(byte function, struct EventStruct *event, String& string)
 float Plugin_013_read(unsigned int taskIndex)
 /*********************************************************************/
 {
-  if (P_013_sensordefs.count(taskIndex) == 0) return 0;
-  if (P_013_sensordefs[taskIndex].sonar == NULL) return 0;
-  delay(1);
-  float distance = P_013_sensordefs[taskIndex].sonar->ping_cm();
-  delay(1);
-  return distance;
+  if (P_013_sensordefs.count(taskIndex) == 0)
+    return 0;
+
+  int16_t max_distance = Settings.TaskDevicePluginConfig[taskIndex][2];
+  int16_t measuringUnit = Settings.TaskDevicePluginConfig[taskIndex][3];
+  int16_t filterType = Settings.TaskDevicePluginConfig[taskIndex][4];
+  int16_t filterSize = Settings.TaskDevicePluginConfig[taskIndex][5];
+  int16_t max_distance_cm = (measuringUnit == UNIT_CM) ? max_distance : (float)max_distance * 2.54f;
+
+  unsigned int echoTime = 0;
+
+  switch  (filterType) {
+    case FILTER_NONE:
+      echoTime = (P_013_sensordefs[taskIndex])->ping();
+      break;
+    case FILTER_MEDIAN:
+      echoTime = (P_013_sensordefs[taskIndex])->ping_median(filterSize, max_distance_cm);
+      break;
+    default:
+      addLog(LOG_LEVEL_INFO, F("invalid Filter Type setting!"));
+  }
+
+  if (measuringUnit == UNIT_CM)
+    return NewPingESP8266::convert_cm_F(echoTime);
+  else
+    return NewPingESP8266::convert_in_F(echoTime);
+}
+
+/*********************************************************************/
+String Plugin_013_getErrorStatusString(unsigned int taskIndex)
+/*********************************************************************/
+{
+  if (P_013_sensordefs.count(taskIndex) == 0)
+    return String(F("invalid taskindex"));
+
+  switch ((P_013_sensordefs[taskIndex])->getErrorState()) {
+    case NewPingESP8266::STATUS_SENSOR_READY: {
+      return String(F("Sensor ready"));
+      break;
+    }
+
+    case NewPingESP8266::STATUS_MEASUREMENT_VALID: {
+      return String(F("no error, measurement valid"));
+      break;
+    }
+
+    case NewPingESP8266::STATUS_ECHO_TRIGGERED: {
+      return String(F("Echo triggered, waiting for Echo end"));
+      break;
+    }
+
+    case NewPingESP8266::STATUS_ECHO_STATE_ERROR: {
+      return String(F("Echo pulse error, Echopin not low on trigger"));
+      break;
+    }
+
+    case NewPingESP8266::STATUS_ECHO_START_TIMEOUT_50ms: {
+      return String(F("Echo timeout error, no echo start whithin 50 ms"));
+      break;
+    }
+
+    case NewPingESP8266::STATUS_ECHO_START_TIMEOUT_DISTANCE: {
+      return String(F("Echo timeout error, no echo start whithin time for max. distance"));
+      break;
+    }
+
+    case NewPingESP8266::STATUS_MAX_DISTANCE_EXCEEDED: {
+      return String(F("Echo too late, maximum distance exceeded"));
+      break;
+    }
+
+    default: {
+      return String(F("unknown error"));
+      break;
+    }
+
+  }
 }
 #endif // USES_P013

src/_P016_IR.ino

@@ -41,6 +41,9 @@ decode_results results;
 #define PLUGIN_NAME_016       "Communication - TSOP4838"
 #define PLUGIN_VALUENAME1_016 "IR"
 
+// A lot of the following code has been taken directly (with permission) from the IRrecvDumpV2.ino example code
+// of the IRremoteESP8266 library. (https://github.com/markszabo/IRremoteESP8266)
+
 // ==================== start of TUNEABLE PARAMETERS ====================
 // As this program is a special purpose capture/decoder, let us use a larger
 // than normal buffer so we can handle Air Conditioner remote codes.
@@ -155,7 +158,10 @@ boolean Plugin_016(byte function, struct EventStruct *event, String& string)
       {
         if (irReceiver->decode(&results))
         {
+          unsigned long IRcode = results.value;
           irReceiver->resume();
+          UserVar[event->BaseVarIndex] = (IRcode & 0xFFFF);
+          UserVar[event->BaseVarIndex + 1] = ((IRcode >> 16) & 0xFFFF);
           String log = "IR: ";
           if (results.overflow)
             log += F("WARNING: IR code is too big for buffer. This result shouldn't be trusted until this is resolved. Edit & increase CAPTURE_BUFFER_SIZE.\n");

src/_P019_PCF8574.ino

@@ -11,7 +11,7 @@
 boolean Plugin_019(byte function, struct EventStruct *event, String& string)
 {
   boolean success = false;
-  static byte switchstate[TASKS_MAX];
+  static boolean switchstate[TASKS_MAX];
 
   switch (function)
   {
@@ -20,10 +20,10 @@ boolean Plugin_019(byte function, struct EventStruct *event, String& string)
       {
         Device[++deviceCount].Number = PLUGIN_ID_019;
         Device[deviceCount].Type = DEVICE_TYPE_I2C;
-        Device[deviceCount].VType = SENSOR_TYPE_SINGLE;
+        Device[deviceCount].VType = SENSOR_TYPE_SWITCH;
         Device[deviceCount].Ports = 8;
         Device[deviceCount].PullUpOption = false;
-        Device[deviceCount].InverseLogicOption = false;
+        Device[deviceCount].InverseLogicOption = true;
         Device[deviceCount].FormulaOption = false;
         Device[deviceCount].ValueCount = 1;
         Device[deviceCount].SendDataOption = true;
@@ -66,10 +66,17 @@ boolean Plugin_019(byte function, struct EventStruct *event, String& string)
         // read and store current state to prevent switching at boot time
         switchstate[event->TaskIndex] = Plugin_019_Read(Settings.TaskDevicePort[event->TaskIndex]);
 
-        // if boot state must be send, inverse default state
-        if (Settings.TaskDevicePluginConfig[event->TaskIndex][0])
-          switchstate[event->TaskIndex] = !switchstate[event->TaskIndex];
+        if (Settings.TaskDevicePin1Inversed[event->TaskIndex]){
+          UserVar[event->BaseVarIndex] = !switchstate[event->TaskIndex];
+        } else {
+          UserVar[event->BaseVarIndex] = switchstate[event->TaskIndex];
+        }
 
+        // if boot state must be send, inverse default state
+        // this is done to force the trigger in PLUGIN_TEN_PER_SECOND
+        if (Settings.TaskDevicePluginConfig[event->TaskIndex][0]) {
+          switchstate[event->TaskIndex] = !switchstate[event->TaskIndex];
+        }
         success = true;
         break;
       }
@@ -85,7 +92,13 @@ boolean Plugin_019(byte function, struct EventStruct *event, String& string)
             log += state;
             addLog(LOG_LEVEL_INFO, log);
             switchstate[event->TaskIndex] = state;
-            UserVar[event->BaseVarIndex] = state;
+
+            if (Settings.TaskDevicePin1Inversed[event->TaskIndex]){
+              UserVar[event->BaseVarIndex] = !switchstate[event->TaskIndex];
+            } else {
+              UserVar[event->BaseVarIndex] = switchstate[event->TaskIndex];
+            }
+
             event->sensorType = SENSOR_TYPE_SWITCH;
             sendData(event);
           }
@@ -128,6 +141,22 @@ boolean Plugin_019(byte function, struct EventStruct *event, String& string)
           SendStatus(event->Source, getPinStateJSON(SEARCH_PIN_STATE, PLUGIN_ID_019, event->Par1, log, 0));
         }
 
+        if (command == F("pcfgpiotoggle"))
+        {
+          success = true;
+          byte mode;
+          uint16_t currentState;
+
+          getPinState(PLUGIN_ID_019, event->Par1, &mode, &currentState);
+          if (mode == PIN_MODE_OUTPUT || mode == PIN_MODE_UNDEFINED) {
+            setPinState(PLUGIN_ID_019, event->Par1, PIN_MODE_OUTPUT, !currentState);
+            Plugin_019_Write(event->Par1, !currentState);
+            log = String(F("PCF  : Toggle GPIO ")) + String(event->Par1) + String(F(" Set to ")) + String(!currentState);
+            addLog(LOG_LEVEL_INFO, log);
+            SendStatus(event->Source, getPinStateJSON(SEARCH_PIN_STATE, PLUGIN_ID_019, event->Par1, log, 0));
+          }
+        }
+
         if (command == F("pcfpulse"))
         {
           success = true;

src/_P028_BME280.ino

@@ -382,8 +382,10 @@ bool Plugin_028_update_measurements(const uint8_t i2cAddress, float tempOffset,
     return false;
   }
 
-  if (!timeOutReached(sensor.last_measurement + 1000)) {
-    // Must wait one second to make sure the filtered values stabilize.
+  // It takes at least 1.587 sec for valit measurements to complete.
+  // The datasheet names this the "T63" moment.
+  // 1 second = 63% of the time needed to perform a measurement.
+  if (!timeOutReached(sensor.last_measurement + 1587)) {
     return false;
   }
   if (!Plugin_028_readUncompensatedData(i2cAddress)) {
@@ -510,7 +512,7 @@ bool Plugin_028_begin(uint8_t i2cAddress) {
   I2C_write8_reg(i2cAddress, BMx280_REGISTER_SOFTRESET, 0xB6);
   delay(2);  // Startup time is 2 ms (datasheet)
   Plugin_028_readCoefficients(i2cAddress);
-  delay(65); //May be needed here as well to fix first wrong measurement?
+//  delay(65); //May be needed here as well to fix first wrong measurement?
   return true;
 }
 
@@ -548,8 +550,8 @@ void Plugin_028_readCoefficients(uint8_t i2cAddress)
 bool Plugin_028_readUncompensatedData(uint8_t i2cAddress) {
   // wait until measurement has been completed, otherwise we would read
   // the values from the last measurement
-  while (I2C_read8_reg(i2cAddress, BMx280_REGISTER_STATUS) & 0x08)
-    delay(1);
+  if (I2C_read8_reg(i2cAddress, BMx280_REGISTER_STATUS) & 0x08)
+    return false;
 
   I2Cdata_bytes BME280_data(BME280_P_T_H_DATA_LEN, BME280_DATA_ADDR);
   bool allDataRead = I2C_read_bytes(i2cAddress, BME280_data);
@@ -642,13 +644,6 @@ float Plugin_028_readHumidity(uint8_t i2cAddress)
     // No support for humidity
     return 0.0;
   }
-  // It takes at least 1.587 sec for valit measurements to complete.
-  // The datasheet names this the "T63" moment.
-  // 1 second = 63% of the time needed to perform a measurement.
-  unsigned long difTime = millis() - sensor.last_measurement;
-  if (difTime < 1587) {
-    delay(1587 - difTime);
-  }
   int32_t adc_H = sensor.uncompensated.humidity;
 
   int32_t v_x1_u32r;

src/_P029_Output.ino

@@ -53,7 +53,7 @@ boolean Plugin_029(byte function, struct EventStruct *event, String& string)
         addHtml(F("<TR><TD>IDX:<TD>"));
         String id = F("TDID");   //="taskdeviceid"
         id += controllerNr + 1;
-        addNumericBox(id, Settings.TaskDeviceID[controllerNr][event->TaskIndex], 0, 9999);
+        addNumericBox(id, Settings.TaskDeviceID[controllerNr][event->TaskIndex], 0, DOMOTICZ_MAX_IDX);
         success = true;
         break;
       }

src/_P035_IRTX.ino

@@ -274,6 +274,8 @@ boolean Plugin_035(byte function, struct EventStruct *event, String& string)
   return success;
 }
 
+// A lot of the following code has been taken directly (with permission) from the IRMQTTServer.ino example code
+// of the IRremoteESP8266 library. (https://github.com/markszabo/IRremoteESP8266)
 
 // Parse an Air Conditioner A/C Hex String/code and send it.
 // Args:
@@ -615,4 +617,4 @@ uint16_t countValuesInStr(const String str, char sep) {
 //  }
 //  return result;
 //}
-#endif // USES_P035
+#endif // USES_P035

src/_P037_MQTTImport.ino

@@ -24,6 +24,37 @@
 WiFiClient espclient_037;
 PubSubClient *MQTTclient_037 = NULL;
 bool MQTTclient_037_connected = false;
+int reconnectCount = 0;
+
+String getClientName() {
+  //
+  // Generate the MQTT import client name from the system name and a suffix
+  //
+  String tmpClientName = F("%sysname%-Import");
+  String ClientName = parseTemplate(tmpClientName, 20);
+  if (reconnectCount != 0) ClientName += reconnectCount;
+  return ClientName;
+}
+
+void Plugin_037_try_connect() {
+  Plugin_037_update_connect_status();
+  if (MQTTclient_037_connected) return;
+  // workaround see: https://github.com/esp8266/Arduino/issues/4497#issuecomment-373023864
+  espclient_037 = WiFiClient();
+  espclient_037.setTimeout(CONTROLLER_CLIENTTIMEOUT_DFLT);
+
+  if (MQTTclient_037 == NULL) {
+    MQTTclient_037 = new PubSubClient(espclient_037);
+  }
+  if (MQTTConnect_037())
+  {
+    //		Subscribe to ALL the topics from ALL instance of this import module
+    MQTTSubscribe_037();
+  } else {
+    MQTTclient_037->disconnect();
+  }
+  Plugin_037_update_connect_status();
+}
 
 void Plugin_037_update_connect_status() {
   bool connected = false;
@@ -39,6 +70,8 @@ void Plugin_037_update_connect_status() {
     if (!connected) {
       // workaround see: https://github.com/esp8266/Arduino/issues/4497#issuecomment-373023864
       espclient_037 = WiFiClient();
+      espclient_037.setTimeout(CONTROLLER_CLIENTTIMEOUT_DFLT);
+      ++reconnectCount;
       addLog(LOG_LEVEL_ERROR, F("IMPT : MQTT 037 Connection lost"));
     }
   }
@@ -49,11 +82,6 @@ boolean Plugin_037(byte function, struct EventStruct *event, String& string)
   boolean success = false;
 
   char deviceTemplate[4][41];		// variable for saving the subscription topics
-  //
-  // Generate the MQTT import client name from the system name and a suffix
-  //
-  String tmpClientName = F("%sysname%-Import");
-  String ClientName = parseTemplate(tmpClientName, 20);
 
   switch (function)
   {
@@ -121,30 +149,23 @@ boolean Plugin_037(byte function, struct EventStruct *event, String& string)
 
     case PLUGIN_INIT:
       {
-        if (!MQTTclient_037) {
-          MQTTclient_037 = new PubSubClient(espclient_037);
-        }
-
+        success = false;
         //    When we edit the subscription data from the webserver, the plugin is called again with init.
         //    In order to resubscribe we have to disconnect and reconnect in order to get rid of any obsolete subscriptions
-
-        MQTTclient_037->disconnect();
-
-        if (MQTTConnect_037(ClientName))
-        {
-          //		Subscribe to ALL the topics from ALL instance of this import module
-          MQTTSubscribe_037();
-          success = true;
-        }
-        else
-        {
-          success = false;
+        if (MQTTclient_037 != NULL) {
+          MQTTclient_037->disconnect();
+          if (MQTTConnect_037())
+          {
+            //		Subscribe to ALL the topics from ALL instance of this import module
+            MQTTSubscribe_037();
+            success = true;
+          }
         }
       }
 
     case PLUGIN_TEN_PER_SECOND:
       {
-        if (!MQTTclient_037->loop()) {		// Listen out for callbacks
+        if (MQTTclient_037 != NULL && !MQTTclient_037->loop()) {		// Listen out for callbacks
           Plugin_037_update_connect_status();
         }
         success = true;
@@ -154,25 +175,26 @@ boolean Plugin_037(byte function, struct EventStruct *event, String& string)
     case PLUGIN_ONCE_A_SECOND:
       {
         //  Here we check that the MQTT client is alive.
+        Plugin_037_try_connect();
+        if (MQTTclient_037 != NULL) {
+          if (!MQTTclient_037->connected() || MQTTclient_should_reconnect) {
+            if (MQTTclient_should_reconnect) {
+              addLog(LOG_LEVEL_ERROR, F("IMPT : MQTT 037 Intentional reconnect"));
+            }
 
-        if (!MQTTclient_037->connected() || MQTTclient_should_reconnect) {
-          if (MQTTclient_should_reconnect) {
-            addLog(LOG_LEVEL_ERROR, F("IMPT : MQTT 037 Intentional reconnect"));
+            MQTTclient_037->disconnect();
+            Plugin_037_update_connect_status();
+            delay(250);
+
+            if (! MQTTConnect_037()) {
+              success = false;
+              break;
+            }
+
+            MQTTSubscribe_037();
           }
-
-          MQTTclient_037->disconnect();
-          Plugin_037_update_connect_status();
-          delay(250);
-
-          if (! MQTTConnect_037(ClientName)) {
-            success = false;
-            break;
-          }
-
-          MQTTSubscribe_037();
+          success = true;
         }
-
-        success = true;
         break;
       }
 
@@ -260,6 +282,7 @@ boolean Plugin_037(byte function, struct EventStruct *event, String& string)
 }
 boolean MQTTSubscribe_037()
 {
+  if (!MQTTclient_037_connected) return false;
 
   // Subscribe to the topics requested by ALL calls to this plugin.
   // We do this because if the connection to the broker is lost, we want to resubscribe for all instances.
@@ -283,7 +306,7 @@ boolean MQTTSubscribe_037()
         if (subscribeTo.length() > 0)
         {
           parseSystemVariables(subscribeTo, false);
-          if (MQTTclient_037->subscribe(subscribeTo.c_str()))
+          if (MQTTclient_037 != NULL && MQTTclient_037->subscribe(subscribeTo.c_str()))
           {
             String log = F("IMPT : [");
             LoadTaskSettings(y);
@@ -350,9 +373,11 @@ void mqttcallback_037(char* c_topic, byte* b_payload, unsigned int length)
 // For some reason this seems to failduring the call in INIT- however it succeeds later during recovery
 // It would be nice to understand this....
 
-boolean MQTTConnect_037(String clientid)
+boolean MQTTConnect_037()
 {
   boolean result = false;
+  if (MQTTclient_037 == NULL) return false;
+  String clientid = getClientName();
   // @ToDo TD-er: Plugin allows for more than one MQTT controller, but we're now using only the first enabled one.
   int enabledMqttController = firstEnabledMQTTController();
   if (enabledMqttController < 0) {
@@ -368,7 +393,7 @@ boolean MQTTConnect_037(String clientid)
     return false; // Not connected, so no use in wasting time to connect to a host.
   }
   ControllerSettingsStruct ControllerSettings;
-  LoadControllerSettings(enabledMqttController, (byte*)&ControllerSettings, sizeof(ControllerSettings));
+  LoadControllerSettings(enabledMqttController, ControllerSettings);
   if (ControllerSettings.UseDNS) {
     MQTTclient_037->setServer(ControllerSettings.getHost().c_str(), ControllerSettings.Port);
   } else {

src/_P044_P1WifiGateway.ino

@@ -16,7 +16,7 @@
 
 #define P044_STATUS_LED 12
 #define P044_BUFFER_SIZE 1024
-#define P044_NETBUF_SIZE 600
+#define P044_NETBUF_SIZE 128
 #define P044_DISABLED 0
 #define P044_WAITING 1
 #define P044_READING 2
@@ -31,7 +31,7 @@ boolean CRCcheck = false;
 unsigned int currCRC = 0;
 int checkI = 0;
 
-WiFiServer *P1GatewayServer;
+WiFiServer *P1GatewayServer = nullptr;
 WiFiClient P1GatewayClient;
 
 boolean Plugin_044(byte function, struct EventStruct *event, String& string)
@@ -125,7 +125,11 @@ boolean Plugin_044(byte function, struct EventStruct *event, String& string)
           #if defined(ESP32)
             Serial.begin(ExtraTaskSettings.TaskDevicePluginConfigLong[1], serialconfig);
           #endif
-          if (P1GatewayServer) P1GatewayServer->close();
+          if (P1GatewayServer)
+          {
+            P1GatewayServer->close();
+            delete P1GatewayServer;
+          }
           P1GatewayServer = new WiFiServer(ExtraTaskSettings.TaskDevicePluginConfigLong[0]);
           P1GatewayServer->begin();
 
@@ -164,7 +168,7 @@ boolean Plugin_044(byte function, struct EventStruct *event, String& string)
       {
         if (P1GatewayServer) {
           P1GatewayServer->close();
-          //FIXME: shouldnt P1P1GatewayServer be deleted?
+          delete P1GatewayServer;
           P1GatewayServer = NULL;
         }
         success = true;
@@ -175,7 +179,6 @@ boolean Plugin_044(byte function, struct EventStruct *event, String& string)
       {
         if (Plugin_044_init)
         {
-          size_t bytes_read;
           if (P1GatewayServer->hasClient())
           {
             if (P1GatewayClient) P1GatewayClient.stop();
@@ -186,24 +189,25 @@ boolean Plugin_044(byte function, struct EventStruct *event, String& string)
           if (P1GatewayClient.connected())
           {
             connectionState = 1;
-            uint8_t net_buf[P044_BUFFER_SIZE];
+            uint8_t net_buf[P044_NETBUF_SIZE];
             int count = P1GatewayClient.available();
             if (count > 0)
             {
-              if (count > P044_BUFFER_SIZE)
-                count = P044_BUFFER_SIZE;
-              bytes_read = P1GatewayClient.read(net_buf, count);
-              Serial.write(net_buf, bytes_read);
+              size_t net_bytes_read;
+              if (count > P044_NETBUF_SIZE)
+                count = P044_NETBUF_SIZE;
+              net_bytes_read = P1GatewayClient.read(net_buf, count);
+              Serial.write(net_buf, net_bytes_read);
               Serial.flush(); // Waits for the transmission of outgoing serial data to complete
 
-              if (count == P044_BUFFER_SIZE) // if we have a full buffer, drop the last position to stuff with string end marker
+              if (count == P044_NETBUF_SIZE) // if we have a full buffer, drop the last position to stuff with string end marker
               {
                 count--;
                 // and log buffer full situation
                 addLog(LOG_LEVEL_ERROR, F("P1   : Error: network buffer full!"));
               }
               net_buf[count] = 0; // before logging as a char array, zero terminate the last position to be safe.
-              char log[P044_BUFFER_SIZE + 40];
+              char log[P044_NETBUF_SIZE + 40];
               sprintf_P(log, PSTR("P1   : Error: N>: %s"), (char*)net_buf);
               addLog(LOG_LEVEL_DEBUG, log);
             }

src/_P073_7DGT.ino

@@ -458,6 +458,7 @@ void p073_FillBufferWithTemp(long temperature)
   int p073_numlenght = strlen(p073_digit);
   byte p073_dispdigit;
   for (int i=0;i<p073_numlenght;i++) {
+    p073_dispdigit = 10;           // default is space
     if (p073_digit[i] > 47 && p073_digit[i] < 58)
       p073_dispdigit = p073_digit[i]-48;
     else if (p073_digit[i] == 32)  // space

src/_P075_Nextion.ino

@@ -3,10 +3,15 @@
 //################################### Plugin 075: Nextion <info@sensorio.cz>  ###########################
 //###################################   Created on the work of  majklovec     ###########################
 //###################################    Revisions by BertB and ThomasB       ########################### 
-//###################################    Last Revision: Aug-03-2018 (TB)      ###########################
+//###################################    Last Revision: Oct-03-2018 (TB)      ###########################
 //#######################################################################################################
 //
-// Updated: Aug-03-2018, ThomasB.
+// Updated: Oct-03-2018, ThomasB.
+// Added DEBUG_LOG define to reduce info log messages and prevent serial log flooding.
+// Added SendStatus() to post log message on browser to acknowledge HTTP write.
+// Added reserve() to minimize string memory allocations.
+//
+// Aug-03-2018 change log summary:
 // This updated Nextion Plugin introduces hardware (UART) serial as an alternative to SoftSerial.
 // Hardware serial offers more reliable communication with the Nextion display.
 // Created on the experimental Serial.swap() work of BertB.
@@ -19,6 +24,9 @@
 // Defines start here
 // *****************************************************************************************************
 
+//#define DEBUG_LOG             // Enable this to include additional info messages in log output.
+
+
 // Plug-In defines
 #define PLUGIN_075
 #define PLUGIN_ID_075 75
@@ -241,7 +249,9 @@ boolean Plugin_075(byte function, struct EventStruct *event, String& string)
         SoftSerial = NULL;
       }
 
-      String log = F("NEXTION075 : serial pin config RX:");
+      String log;
+      log.reserve(80);                                 // Prevent re-allocation
+      log = F("NEXTION075 : serial pin config RX:");
       log += rxPin;
       log += F(", TX:");
       log += txPin;
@@ -313,6 +323,7 @@ boolean Plugin_075(byte function, struct EventStruct *event, String& string)
             RssiIndex = UcTmpString.indexOf(F("RSSIBAR"));  // RSSI bargraph Keyword found, wifi value in dBm.
             if(RssiIndex >= 0) {
               int barVal;
+              newString.reserve(Lenlines+10);               // Prevent re-allocation
               newString = tmpString.substring(0, RssiIndex);
               int nbars = WiFi.RSSI();
               if (nbars < -100 || nbars >= 0)
@@ -345,31 +356,38 @@ boolean Plugin_075(byte function, struct EventStruct *event, String& string)
             }
 
             sendCommand(newString.c_str(), HwSerial);
-            
-            String log = F("NEXTION075 : Cmd Statement Line-");
-            log += String(x+1);
-            log += F(" Sent: ");
-            log += newString;
-            addLog(LOG_LEVEL_INFO, log);
+            #ifdef DEBUG_LOG 
+              String log;
+              log.reserve(Lenlines+50);                 // Prevent re-allocation
+              log = F("NEXTION075 : Cmd Statement Line-");
+              log += String(x+1);
+              log += F(" Sent: ");
+              log += newString;
+              addLog(LOG_LEVEL_INFO, log);
+            #endif
           }
         }
 
         // At Interval timer, send idx & value data only if user enabled "values" interval mode.
         if(IncludeValues) {
-/* 
-            String log = F("NEXTION075: Interval values data enabled, resending idx=");
-            log += String(UserVar[event->BaseVarIndex]);
-            log += F(", value=");
-            log += String(UserVar[event->BaseVarIndex+1]);
-            addLog(LOG_LEVEL_INFO, log);
-*/
+            #ifdef DEBUG_LOG
+             String log;
+             log.reserve(120);                          // Prevent re-allocation
+             log = F("NEXTION075: Interval values data enabled, resending idx=");
+             log += String(UserVar[event->BaseVarIndex]);
+             log += F(", value=");
+             log += String(UserVar[event->BaseVarIndex+1]);
+             addLog(LOG_LEVEL_INFO, log);
+            #endif
+
             success = true;
         }
         else {
-/*
-            String log = F("NEXTION075: Interval values data disabled, idx & value not resent");
-            addLog(LOG_LEVEL_INFO, log);
-*/
+            #ifdef DEBUG_LOG
+             String log = F("NEXTION075: Interval values data disabled, idx & value not resent");
+             addLog(LOG_LEVEL_INFO, log);
+            #endif
+
             success = false;
         } 
 
@@ -386,6 +404,8 @@ boolean Plugin_075(byte function, struct EventStruct *event, String& string)
         
 // Enable addLog() code below to help debug plugin write problems.
 /*
+        String log;
+        log.reserve(140);                               // Prevent re-allocation
         String log = F("Nextion arg0: ");
         log += tmpString; 
         log += F(", TaskDeviceName: ");
@@ -401,11 +421,14 @@ boolean Plugin_075(byte function, struct EventStruct *event, String& string)
             tmpString = string.substring(argIndex + 1);
             sendCommand(tmpString.c_str(), HwSerial);
 
-            String log = F("NEXTION075 : WRITE, ");
-            log += F("Command is ");
-            log += (tmpString.c_str());
-            addLog(LOG_LEVEL_INFO, log);
-
+            String log;
+            log.reserve(110);                           // Prevent re-allocation
+            log = F("NEXTION075 : WRITE = ");
+            log += tmpString;
+            #ifdef DEBUG_LOG  
+              addLog(LOG_LEVEL_INFO, log);
+            #endif
+            SendStatus(event->Source, log);             // Reply (echo) to sender. This will print message on browser.
             success = true;                             // Set true only if plugin found a command to execute.
         }
         break;
@@ -439,7 +462,6 @@ boolean Plugin_075(byte function, struct EventStruct *event, String& string)
       uint16_t i;
       uint8_t c;
       uint8_t charCount;
-      String log;
       String Vidx;
       String Nvalue;
       String Svalue;
@@ -447,7 +469,7 @@ boolean Plugin_075(byte function, struct EventStruct *event, String& string)
       char __buffer[RXBUFFSZ+1];                        // Staging buffer.
       
       if(rxPin < 0) {
-        String log = F("NEXTION075 : Missing RxD Pin Number, aborted serial receive");
+        String log = F("NEXTION075 : Missing RxD Pin, aborted serial receive");
         addLog(LOG_LEVEL_INFO, log);
         break;
       }
@@ -455,6 +477,8 @@ boolean Plugin_075(byte function, struct EventStruct *event, String& string)
       if(HwSerial == UARTSERIAL) {
         charCount = Serial.available();                 // Prime the Hardware Serial engine.
         if(charCount >= RXBUFFWARN) {                   // ESP8266 has 128 byte circular Rx buffer.
+            String log;
+            log.reserve(70);                           // Prevent re-allocation
             log = F("NEXTION075 : RxD UART Buffer capacity warning,");
             log += String(charCount);
             log += F(" bytes");
@@ -465,6 +489,8 @@ boolean Plugin_075(byte function, struct EventStruct *event, String& string)
         if(SoftSerial == NULL) break;                   // SoftSerial missing, exit.
         charCount = SoftSerial->available();            // Prime the Soft Serial engine.
         if(charCount >= RXBUFFWARN) {
+            String log;
+            log.reserve(70);                           // Prevent re-allocation
             log = F("NEXTION075 : RxD SoftSerial Buffer capacity warning, ");
             log += String(charCount);
             log += F(" bytes");
@@ -495,13 +521,17 @@ boolean Plugin_075(byte function, struct EventStruct *event, String& string)
               UserVar[event->BaseVarIndex + 1] = __buffer[3];
               sendData(event);
 
-              log = F("NEXTION075 : code: ");
-              log += __buffer[1];
-              log += ",";
-              log += __buffer[2];
-              log += ",";
-              log += __buffer[3];
-              addLog(LOG_LEVEL_INFO, log);
+              #ifdef DEBUG_LOG
+                String log;
+                log.reserve(70);                        // Prevent re-allocation
+                log = F("NEXTION075 : code: ");
+                log += __buffer[1];
+                log += ",";
+                log += __buffer[2];
+                log += ",";
+                log += __buffer[3];
+                addLog(LOG_LEVEL_INFO, log);
+              #endif
             }
           }
         } 
@@ -534,9 +564,14 @@ boolean Plugin_075(byte function, struct EventStruct *event, String& string)
             __buffer[i] = 0x00;
             
             String tmpString = __buffer;
-            log = F("NEXTION075 : code: ");
-            log += tmpString;
-            addLog(LOG_LEVEL_INFO, log);
+
+            #ifdef DEBUG_LOG
+              String log;
+              log.reserve(50);                          // Prevent re-allocation
+              log = F("NEXTION075 : Code = ");
+              log += tmpString;
+              addLog(LOG_LEVEL_INFO, log);
+            #endif
 
             int argIndex = tmpString.indexOf(F(",i"));
             int argEnd = tmpString.indexOf(',', argIndex + 1);
@@ -568,14 +603,21 @@ boolean Plugin_075(byte function, struct EventStruct *event, String& string)
                 UserVar[event->BaseVarIndex+1] = Svalue.toFloat();
                 sendData(event);
 
-                log = F("NEXTION075 : Pipe Command Sent: ");
-                log += __buffer;
-                log += UserVar[event->BaseVarIndex];
+                #ifdef DEBUG_LOG
+                 String log;
+                 log.reserve(80);                       // Prevent re-allocation
+                 log = F("NEXTION075 : Pipe Command Sent: ");
+                 log += __buffer;
+                 log += UserVar[event->BaseVarIndex];
+                 addLog(LOG_LEVEL_INFO, log);
+                #endif
             }
             else {
-                log = F("NEXTION075 : Unknown Pipe Command, skipped");
+                #ifdef DEBUG_LOG
+                 String log = F("NEXTION075 : Unknown Pipe Command, skipped");
+                 addLog(LOG_LEVEL_INFO, log);
+                #endif
             }
-            addLog(LOG_LEVEL_INFO, log);
           }
         }
         if(HwSerial == UARTSERIAL) charCount = Serial.available();

src/__CPlugin.ino

@@ -148,11 +148,19 @@ byte CPluginCall(byte Function, struct EventStruct *event)
   {
     // Unconditional calls to all plugins
     case CPLUGIN_PROTOCOL_ADD:
-      for (x = 0; x < CPLUGIN_MAX; x++)
-        if (CPlugin_id[x] != 0){
+      for (x = 0; x < CPLUGIN_MAX; x++) {
+        if (CPlugin_id[x] != 0) {
+          const unsigned int next_ProtocolIndex = protocolCount + 2;
+          if (next_ProtocolIndex > Protocol.size()) {
+            // Increase with 8 to get some compromise between number of resizes and wasted space
+            unsigned int newSize = Protocol.size();
+            newSize = newSize + 8 - (newSize % 8);
+            Protocol.resize(newSize);
+          }
           checkRAM(F("CPluginCallADD"),x);
           CPlugin_ptr[x](Function, event, dummyString);
         }
+      }
       return true;
       break;
 

src/__Plugin.ino

@@ -1113,6 +1113,15 @@ byte PluginCall(byte Function, struct EventStruct *event, String& str)
     case PLUGIN_UNCONDITIONAL_POLL:
       for (byte x = 0; x < PLUGIN_MAX; x++) {
         if (Plugin_id[x] != 0){
+          if (Function == PLUGIN_DEVICE_ADD) {
+            const unsigned int next_DeviceIndex = deviceCount + 2;
+            if (next_DeviceIndex > Device.size()) {
+              // Increase with 16 to get some compromise between number of resizes and wasted space
+              unsigned int newSize = Device.size();
+              newSize = newSize + 16 - (newSize % 16);
+              Device.resize(newSize);
+            }
+          }
           START_TIMER;
           Plugin_ptr[x](Function, event, str);
           STOP_TIMER_TASK(x,Function);
@@ -1243,11 +1252,10 @@ byte PluginCall(byte Function, struct EventStruct *event, String& str)
             // Schedule the plugin to be read.
             schedule_task_device_timer_at_init(event->TaskIndex);
           }
-          if (Function != PLUGIN_GET_DEVICEVALUENAMES) {
+          if (ExtraTaskSettings.TaskIndex != event->TaskIndex) {
             // LoadTaskSettings may call PLUGIN_GET_DEVICEVALUENAMES.
             LoadTaskSettings(event->TaskIndex);
           }
-
           event->BaseVarIndex = event->TaskIndex * VARS_PER_TASK;
           checkRAM(F("PluginCall_init"),x);
           START_TIMER;

src/define_plugin_sets.h

@@ -206,10 +206,25 @@ To create/register a plugin, you have to :
     #define USES_P036   // FrameOLED
     #define USES_P027   // INA219
     #define USES_P028   // BME280
-
-    // TODO : Add list of compatible plugins for ESP32 board.
 #endif
 
+#ifdef PLUGIN_SET_TEST_ESP32
+    #ifndef ESP32
+        #define ESP32
+    #endif
+    #ifdef ESP8266
+        #undef ESP8266
+    #endif
+//    #define PLUGIN_SET_ONLY_SWITCH
+
+    #define  PLUGIN_SET_TESTING
+    #define  CONTROLLER_SET_STABLE
+    #define  NOTIFIER_SET_STABLE
+    #define  PLUGIN_SET_STABLE     // add stable
+    // See also PLUGIN_SET_TEST_ESP32 section at end,
+    // where incompatible plugins will be disabled.
+    // TODO : Check compatibility of plugins for ESP32 board.
+#endif
 
 
 // Generic ------------------------------------
@@ -585,8 +600,22 @@ To create/register a plugin, you have to :
 #ifdef NOTIFIER_SET_EXPERIMENTAL
 #endif
 
+/******************************************************************************\
+ * Remove incompatible plugins ************************************************
+\******************************************************************************/
+#ifdef PLUGIN_SET_TEST_ESP32
+  #undef USES_P010   // BH1750          (doesn't work yet on ESP32)
+  #undef USES_P049   // MHZ19           (doesn't work yet on ESP32)
 
+  #undef USES_P052   // SenseAir        (doesn't work yet on ESP32)
+  #undef USES_P053   // PMSx003
 
+  #undef USES_P056   // SDS011-Dust     (doesn't work yet on ESP32)
+  #undef USES_P065   // DRF0299
+  #undef USES_P071   // Kamstrup401
+  #undef USES_P075   // Nextion
+  #undef USES_P078   // Eastron Modbus Energy meters (doesn't work yet on ESP32)
+#endif
 
 
 /******************************************************************************\