What is Lucid?

Lucid is a lightweight communications protocol based on open, widely adopted internet standards like JSON and MQTT. In its basic form it provides a standards-based, easy to use way of sharing status, configuration, alarms and data values between devices and systems over a network. At a high level:

• JSON (JavaScript Object Notation) is a way of presenting structured data, allowing different systems to interpret the data in the same way. It can be considered a much lighter cousin to XML.
• MQTT (Message Queue Telemetry Transport) is an extremely lightweight publish/subscribe messaging transport used to connect different systems together and especially IoT like devices to servers.

Lucid calls remote devices “Field Devices” (FDs); server applications like SCADA, Telemetry Systems and cloud Data Warehouses are called “Supervisory Applications” (SAs).  Lucid can be used as the link between Field Devices (FDs) and Supervisory Applications (SAs) (as in a traditional SCADA system) or as a link between different Supervisory Applications (SAs).  Because Lucid is specifically designed for Industrial IoT (IIOT) and Operational Technology (OT) systems, it is excellent at providing a way for multiple entities to share telemetry data and coordinate configuration of telemetry devices.

Lucid provides a number of functions above and beyond the reporting of data values. Some of the most important of these are:

• Configuration – FDs report their configuration back to SAs so that the configuration of a device is reliably known. SAs can also share configuration and changes with other SAs.  The protocol supports SAs changing this configuration remotely – either directly, or with SAs percolating configuration changes to the SAs that can send the changes to the FDs.
• Point states, creating events and alarms – a consistent model of alarming and eventing is mandated by the protocol so every FD uses that same model. Once a user has learned the model for one type of FD they will know it for all FDs.
• Logging – FDs can be capable of logging point values for collection at a later time. This allows the number of contacts with the SA to be minimised and battery life to be preserved.
• Extended Points – FDs can support the generation of points which represent values derived from other points values. For example, an extended point could represent the average value of another point, or the maximum or the minimum. Alternatively, the extended point could represent the number of times a state has changed for a point (e.g. pump goes from off to on) or the time for which a point has been in a certain state (e.g. pump hours run)
• Security – By default Lucid can be used with TLS encryption switched on for the MQTT links, a standard method of protecting MQTT data. In future releases a more comprehensive end-to-end security capability will be introduced.

Many of these functions are optional for FDs to implement, leaving a simple, easy-to-implement protocol that can be readily incorporated into a vendor’s device. This permits lower cost devices to be made readily available but using a consistent single interface.

What is Lucid similar to?

Lucid is a protocol detailing how devices in the field should talk to Operational Technology systems. In that respect it is like some of the protocols below, which you may have heard of:

• WITS-DNP3
• Modbus
• EthernetIP from Rockwell
• Medina from Metasphere
• Proteus from Schneider

However, its MQTT element is different to the above and in that way, it is more like one of the following:

• Sparkplug
• OPC-UA, in particular the Pub-Sub variant of OPC-UA.

Articles have been prepared which contrasts Lucid against WITS-DNP3 and compare Sparkplug and OPC-UA against Lucid. In essence, Lucid provides very similar functionality to WITS-DNP3 and more functionality that both Sparkplug and OPC-UA. One might ask “if the functionality of WITS-DNP3 and Lucid are similar then why create Lucid?”. The answer to that question is explained in the article on Lucid being lighter than WITS-DNP3.

A further advantage of using the MQTT model is that the interface can be used in many more scenarios that just sending data directly from the FD to a SA; the architecture supports system-to-system interfaces for FD data. See the article on usage scenarios for Lucid for more information.

Proprietary and open interfaces

Some of the protocols mentioned above are proprietary, controlled by one vendor and normally restricted to use on their devices. Others are open but one must pay for access to the protocol. The final set of protocols are totally open and permit anyone to look at and evaluate the protocols. These tend to be more popular due to their easy accessibility.

The Lucid protocol is open and free to access for anyone. From a user’s perspective, this does not translate to free to implement! Users will still have to bear the cost of procuring and supporting systems which support Lucid, even if they are just extensions to current licensing models for existing Master Stations or Field Devices. It is hoped that free and open accessibility will however increase the number of vendors providing Lucid solutions and that in turn will provide good variation in the devices available and healthy competition between the vendors. This is expected to translate to more choice and keener pricing for users.

The big advantage of this open availability is that it is easy for vendors to read and assess the standard. This makes it more likely that they will implement the protocol on their products which would increase the number of vendors supplying compatible devices. The knock-on effect of this is that users should have an increased choice of devices which are compatible with the rest of their OT Infrastructure. This increased choice benefits users as greater competition should result in better value products and greater diversity in the functionality of field devices and supervisory applications.

An open protocol also helps in controlling proliferation of interfaces at the Enterprise level for user organisations. Rather than having to control and maintain many interfaces with many different products, users can request that interfaces use Lucid, meaning they need only deal with one interface. Actively managing the number of interfaces in this method means there are fewer interfaces, with less technical and management overhead, better staff learning and consequently easier deployment and troubleshooting.

Easy to implement

An important factor in designing Lucid was that it should be easy for vendors to implement, thereby reducing barriers for adoption to a minimum. Lucid is free to use and based on heavily standardised, commonly available open-source technologies; vendor’s implementation costs are therefore minimised.

The protocol’s simplicity is expected to encourage implementation. The various permitted levels of protocol compliance (do some things but not others) are expected to encourage implementation from the simplest devices upwards. Implementation on the simplest devices allows mass deployment on distribution networks/infrastructure to support monitoring of those networks/infrastructure. The protocol has been designed to support long battery life, low transmission bandwidth and minimal configuration costs on such devices.

What types of ‘top end’ could be used with Lucid?

Lucid FDs provide information to an MQTT broker as an intermediary between the FD and the SA. Any SCADA, or telemetry or top end software that is capable of ingesting MQTT based data, should be able to work with Lucid. Note however that these systems may require some alteration to deal with the details of the Lucid protocol.

Examples of the kinds of systems that we might expect to support Lucid are:

• Telemetry systems such as GeoSCADA (Schneider Electric), or Scope X (Ovarro)
• Device specific top ends such as WaterCore (Technolog), Palette (Metasphere)
• Other data collection systems such as Ignition (Inductive Automation)
• Bespoke systems

There is no implication in the above that these systems do already support Lucid, just that they could. The list above is just meant to give an idea of the wide scope of implementations that are possible. Loosening the link between FDs and the OT systems which normally consume their data allows users to consume the data directly in their private cloud applications and could avoid users having to build in access to other proprietary and siloed systems.

Introduction to service and testing

Testing of a new protocol and its integration into a user’s environment can be a difficult and time-consuming affair that often must be repeated when new versions of the protocol or device become available. It is intended that Lucid will be accompanied by some form of community-led testing suite. This testing suite would allow any user or vendor to carry out tests against an FD and ensure it is compliant with the protocol. It is hoped that this will promote greater interoperability. Not only could vendors use the testing suite to check their products, but users could use the same suite to check those test results and/or enhance the suite for their specific test cases.

This should simplify and speed up the introduction to service and support ongoing testing when new versions are released to check that they work before deployment.

A chance to be involved with the continued design of the protocol

Unlike some other protocols, organisations have the chance to be involved in the continued development and improvement of the protocol by joining the WITS-PSA. The fact that the protocol is managed by a cross vendor and user body (the WITS-PSA) means that the protocol maintains its vendor agnostic stance with no lock-in to a particular vendor

How would a user organisation use Lucid?

There is no one way to start using Lucid within an organisation, but taking the easy approach, users would select products from their vendors which implement Lucid. After selecting the products, users and vendors would have to make sure the products interoperated successfully. In the past this may have been a sometime long and drawn-out process. We expect that the simplification of configuration handling and automated testing that we plan to put in place, will considerably ease this process.

Looking beyond this first level of implementation, users wanting to do more with the data could prepare types of SA to consume appropriate data from the FDs within their estate, possibly sharing this with other enterprise applications.

What are the main benefits of using Lucid?

Some of the big benefits of using Lucid are:

• Because the protocol is vendor agnostic, free to access and relatively easy to implement, vendors can be encouraged to use the protocol within their devices or systems.
• Having a single interface type to many different devices from different vendors gives users a much easier enterprise architecture problem to solve. Each interface type will carry an implementation cost, support cost and sometimes legacy problems. One interface instead of many different interfaces is preferred.
• The simplicity of the protocol permits it to be used on very small and constrained devices, these are often less costly and permit wider rollout of monitoring within the estate.
• The protocol supports flexible implementation with a business. It can for instance, be used directly between servers and is not restricted to just Field Device (FD) to Supervisory Application (SA). This can be useful in efficiently resolving issues with existing proprietary interfaces.
• The support for configuration of devices within the protocol and synchronisation of that configuration with the SA, provides a cost benefit and data integrity benefit to users who no longer have to keep configuration manually in synch

Summary

The Lucid protocol allows Field Devices to communicate with Supervisory Applications using MQTT. It provides very similar functionality to WITS-DNP3. The protocol can be considered a close neighbour to Sparkplug B and OPC UA, although as presented in other articles it provides a richer functional experience that those protocols, especially in relation to configuration.

The protocol supports implementation on constrained IoT like devices. Its reliance on publicly available and popular standards like MQTT and JSON simplifies implementation for vendors who also have access to good tooling and implementation support for the protocol.

The protocol opens up a number of ways to integrate it with a user’s top end systems so there need be no reliance on specific systems. The ability to use the same interface for many different vendors systems and devices also simplifies a user’s enterprise architecture experience.

Hopefully this has whetted your appetite for trying Lucid!

Dave Howarth (Northumbrian Water) and Mark Davison (Terzo Digital)

July 2024

Please see our Lucid reading page for a collated list of other articles on Lucid.