Last week I've received an email with an interesting set of questions about the Devil Framework and I've learned that all email posted by our mail server is mysteriously considered spam by Yahoo!.

Let's start with a question from me: does any one know how to remove an IP address (X-YahooFilteredBulk: 66.139.77.213) from Yahoo's spam list?

Lets go on with the other question. (PS: I've not included the name of the person who sent the very interesting email for privacy reasons, but you know who you are. Thank you.)

First, the hard question. Is the release weeks, months, or years away?

Short answer: months (3 or 4, I hope).

Long answer: the product is finished and already operating at client plants, but it lacks enough documentation to make it usable for development by others except its creators (and sometimes we discover functionalities we have forgotten, shame on us). I'm working on it right now.

Now the easy questions. Are there any restrictions on the packages one can use in the plugins?

No, a plugin can install external libraries and extension of any. If you can build a wrapper to them, and you can legally use them, you can certainly add them to the framework using a plugin or a library module. Indeed almost all the system extensions (like SNMP, Ethernet/serial PLC interfaces, non-Qt widgets, etc.) are provided via plugins.

And no, the framework has no licensing restrictions against it.

What is required for resources to run the system? Memory? Hard Drive? Minimum CPU?

Off the shelf disk space usage is around ~100Mb for the server instances and ~120Mb for the client (size varies by platform distribution, with the Mac OS X one been the fattest of all with ~180Mb for the client).

For what concerns CPU and memory it really depends on what you are going to do with the framework.

More RAM you have the better: we have servers performing data mining jobs that require over 2Gb RAM for in memory data processing. Usually the server occupies a minimum of ~300Mb, the console a minimum of ~150-200Mb. Python is not very memory friendly and it tends to occupy large chunks of it before releasing any (anyhow it worths the trouble). Memory is cheep, buy as much as you can.

Any modern low-end single core CPU should be enough for basic data processing needs. As before, this depends on what you are going to do and how you are going to do it: Python is an interpreted language and for intensive data processing it requires more CPU cycles than, say, C: if you need top speed it's better to implement a Python wrapper around a C library. You can also take advantage of your shiny multi-core multi-processor system, despite the (in)famous Python GIL, using a simple module we have developed that lets you transparently run and query objects in separate Python processes.

Have you done any throughput tests? For the sake of discussion, if you had a system with 10 Collector nodes and 5 IceBridge Consoles, what is the order of magnitude for the number of events per second that could be handled.

When we made throughput tests for the Events sub-system we achieved ~3000-4000 events/sec per server node on old AMD Athlon 3200 CPUs. As always this depends on what you are doing on the collected events. Event pre-processing, post-processing and archival can drastically change performances (if you apply a Fourier transform on a set of events to generate another event that triggers some actions and is used for situations detection with other thousand of events, you can imagine that performances decrease a lot).

The IceBridge console has a different set of problems (and solutions).

The basic design concept around which the whole system is developed is that almost all event processing and manipulation has to be performed at the nodes nearest to the event source, with the upstream event flux containing only data meaningful to the "higher" layers.

An example may be useful: lets say we have a set of PLC that are queried by some Collectors managed by a central MCP server.

• PLCs "generate" row data about the status of a device and control some of its operational aspects.

• Collectors query PLCs and internal events with collected data are generated and processed locally. This Collectors are configured to not propagate this events to parent nodes. But, if some event "value" is above a configured alarm threshold an "alarm" event is generated and sent upstream. Moreover every 5 minutes an event containing "average" stats about the device is posted.

• The MCP node receives alarm/stats events from child Collectors. Some kind of processing is performed on incoming events and at regular times the child Collectors/PLCs are queried about some kind of information. At scheduled times the PLCs are reconfigured to change operational behavior.

• Plant operators are controlling plant machinery using IceBridge consoles connected to some specific Collector. They can control specific tools and receive alarms of interest. The plant CTO controls the general plant status from a console connected to the MCP. He can also control and override instructions provided by other operators to any connected machinery. The CEO is automatically alerted about production, storage and supply problems by a console connected to the MCP. The views all this people are using do not directly receive all the events generated by the Application Server infrastructure, but only the alarm events they are interest into. When they want to control some specific PLC a specific query is performed to the Tags System that manages the PLC itself.

The Tags System is a plugin that implements "abstracted tags". If you are familiar with SCADA software like WinCC and friends you already know what tags are. Otherwise, just consider a tag as a "smart" variable that gets (and sets) some data from a "device", processes it, archives it (using multiple consolidation functions, time resolutions and retention time spans) and generates "update" events the are propagated to selected parents.

The Tags System is fully integrated into the Devil Framework 's distributed architecture, so you can query any tag from any console or parent node with code like "env.api.tags_system.tags_managers['PLC_1'].tags['auto_destruction_status'].get ()" or using one of the provided widgets.

We have created tag providers for PLCs, custom refrigerator devices, SNMP, Devil Framework's internal system monitor, custom Python code, etc. Creating a new tag provider is quite easy and you can do it in a hundred lines of code.

Could each IceBridge console handle 2000-5000 events per second? If one created widgets that monitored and displayed u to 20 pieces of data per object read from an attached device and then created a view with 200 of these widgets on it, would it bring the system to it's knees or would that be reasonable?

I've not done any test like this but I think that updating so much data would slow down things quite a lot (to say the least). As I said above it's really better to use the distributed system to perform the processing and to selectively query/be notified for/of updates.

Your blog entries describe the IceBridge Console as a viewer for data being collected and processed from the other nodes like a SCADA system or a Distributed Control System Console. Could you foresee any problems running that in reverse?

We already have widgets that can be directly binded to tags, so in example, you can have a slider connected to a tag and when you move it the tag value is updated (and the device connected to it either), an vice-versa.

Is there any reason that someone could not set up widgets that would allow a user to create a graphical model in designer mode and then have the Application Server send data based on that model out to the attached devices?

If you want to know if you can use the Views Editor to directly write something like ladder logic programs, no, you can't. But there's no technical limitation that prevents you to write a widget that lets you develop programs in ladder logic.

Otherwise, if you want to design a form that lets you edit a "production recipe" (tables with some data, flags, etc.) and apply it to to the Application Server and to some of the connected devices, there's no problem at all. Remember that from any console, if you have the right permissions, you can transparently make any call to any node of the system.