There has been progress and things are just about ready to go out for testing. If adding hardware support for one or two devices was the only thing that I were doing that would have been done by now, but since I didn’t want to have to rip out the code for other changes that I wanted to make it’s taking a little longer. There isn’t really anything interesting to write about. It’s pretty straightforward implementation of a design. Just requires that I take the time to write it and that everybody else leaves me alone so I can maintain my focus. That last part can be challenging when you’re writing code in a coffee shop. Every so often I’ll read about some startup that has coders working out of coffee shops (hopefully they get up frequently for another drink or a snack from the bakery case instead of milking a small coffee for eight hours while taking up a table and leeching free WiFi) but a busy coffee shop is really a terrible environment for programming. Too many distractions. It won’t be long before I have something ready for others to test and it shouldn’t be long after that for a proper version 1.4 release. I’m not going to get 2.0 out by the time I had planned to (probably) so I’d like to get as much in the way of features that people were really waiting for into that release as I can.

In unrelated news it seems that my visa for entering Brazil is still valid. I’ll be going back next month traveling through some parts of the country that I haven’t been to previously. The food was really good the last time I was there and it looks like I’ll also be making a return trip to the roasting firm that has the hot pink roaster/afterburner/espresso machine and a bunch of barista competition trophies. The month after I’ll be back in the Pacific Northwest for the Roasters Guild retreat. I’ll probably have photos to share from both of those trips later.

Device configuration in Typica 1.3 was pretty simple… for a programmer. In the script fragment for the logging window you would just instantiate the device, use its methods to configure the measurement channels, and hook those channels up to the various widgets that need the data. One of the changes planned for Typica 2.0 was getting rid that configuration system which. It was amazingly flexible and made it possible to implement new features or try out different ways of doing things much faster, but most people using the software don’t want to muck around with that (this is a completely reasonable position to take). Scriptability isn’t something that’s going away. My intent on the 2.0 branch is to make scripts less visible (no prompt every time the program is started) and more modular. It should be possible to write a file that adds a new feature and share that in such a way that people using the program can just install the new feature to live alongside their current configuration. I still don’t really expect people to do that, but it makes my life easier. Part of that transition requires moving device configuration out of script fragments and into the program proper with configuration widgets that can be adjusted from within the program and this has been partially back-ported to what will be the 1.4 release.

One of the interesting consequences of the current design is a more explicit separation of the devices measurements are coming from and the coffee roaster the devices are used with. Here’s how it works in the case of Modbus:

You have 1 or more Modbus devices in your roaster. These are, possibly through some sort of adapter, connected to your computer and the whole device network shows up as a serial port (COM9 in my case). You want to add these devices to your configuration so in Typica, you go to Device Configuration (don’t look for this in current releases, it’s not there, and don’t trust this with future releases, I’ll make real documentation once there’s a release out), add Modbus RTU devices. First you’ll be able to specify which serial port that’s on. If the devices are connected when you bring this up it will auto-detect all of your serial ports. A lot of laptops don’t have old style serial ports these days and if you don’t have anything else connected, your adapter to these devices might be the only port there is and would already be selected. If you’re not connected but know what port the devices are on, you can just type that in. If you have no clue at all but somehow are able to do the rest, you can just select whatever and change the port settings when you do know what they are. You can also select the baud rate and other details that must be common across all devices on that port. If you carry that laptop around to several roasters and different devices show up on the same port depending on where you are, don’t worry. Typica isn’t going to complain about port conflicts because it’s smart enough (or rather too dumb to care or perhaps just trusting that the person using the software knows what they’re doing and that the software should just shut up and do as its told) to let you configure duplicate ports with different devices.

Now you have your port configured. You probably only have one device on the port (if you have more, just repeat this step until you’ve covered them all), so go ahead and add a device to the port. I’m starting with just a generic device that takes further configuration options but intend to add more options which are built on top of that but provide information that must be configured in the case of generic devices. For example, someone with a Fuji PXR series device will be able to select that, specify the station number for the device, and their device configuration will be finished. In the case of a generic device, however, you’ll need to further specify the relative addresses of data you intend to read or write to the device and how that data should be interpreted. I looked over documentation for several devices and there’s no standardization here even between different lines from the same manufacturer so until someone with the hardware shares working settings with me to incorporate into a release, you’ll have to RTFM for your device and type in a few numbers. Once you figure it out, I’d encourage you to send me teh codez so it will be easier for the next person who wants to use that device. I should probably consider building in a way for the program to produce an email with the relevant information (not in a hidey, sneaky sort of way, but like clicking a mailto: link which opens up your email client, pre-populates a new email, and you can review exactly what would be sent to me and would then need to manually hit that send button) but that’s not an immediate priority for me.

The code for this part is all done. You can add however many devices to the port and it will save the relevant details with QSettings (how that works depends on the platform: on Windows it goes into the registry).

Next, you need to be able to configure a roaster. When doing this, you’ll have access to all of the devices that you’ve configured. You select which ones you want to use and then for each channel that’s configured on those devices you can decide if you get a panel that shows the most recent measurement, if it goes to the table view, if it goes to the graph, that sort of thing. This is also where you can specify what you want in the way of annotation controls. Don’t need the ones I’ve previously put in the example configurations? You can get rid of them. Want ones that I don’t see the point of (lots of people have expressed interest in having first and second crack annotation buttons, for example)? Go ahead and put them in. It’ll be easy. This is the next thing that I need to work on back porting from the 2.0 branch. Then when you want to roast coffee you’ll need to select the roaster you intend to use and the program will look at how you’ve previously configured that roaster and set up the devices and logging view appropriately. While I don’t recommend it (there are safety and focus issues) you could also go back and indicate that you want to roast on another machine with a non-conflicting device set (2.0 gets the ability to split channels from the same device among multiple concurrently operating machines but I’m not going to put the extra effort in for that right now) and have two logging windows running at the same time, one for each roaster.

Adding Modbus Support to Typica - Day 6

Day 5 was also a no progress day because I was helping to put the shop back together so we’d be ready to reopen today. Here’s a picture of the equipment that I’m using to test what I’m doing. The transformer on the left plugs into the wall and gives me the 24VAC power needed for the controller. Thermocouple wire goes between the controller and the calibrator on the right. This lets me set whatever value I want for the process value. The device between the transformer and the controller is an RS485/RS232 to USB adapter. For cable runs as short as I’m using here RS485 works over just about any wire. In this case I just clipped the ends off an old CAT5 patch cable and used the green pair of wires. Suwako doesn’t have anything to do with all of that. She arrived today as a replacement for the cat key chain that recently fell apart.

Today I went over the documentation for the controller looking for addresses that it might be interesting to read or write from, tested that writes were doing the right thing and that reads were returning the values I expected. This also included some multi-value reads (for example, reading the current PV and SV at the same time or getting both the upper and lower SV limits) since it’s faster to read multiple values at once than to read values from close addresses separately. Everything worked as expected. This ends the step of testing that I understand how to interface with the hardware. I’ve also finished the design of the required code and made good progress on writing it. All the hard parts (not that any of this was particularly challenging) are done, now it’s really just a matter of getting the time to write the rest of the code.

Yesterday I wrote a tiny test program that would write a particular string of digits out over a serial port and display the result that comes back. What went out was 0x010403E80001B1BA which is Modbus for Station 1, read out input registers (command 0x04) starting from relative address 0x03E8 (PV in engineering units) 1 value (0x0001). The 0xB1BA is a check digit computed from everything in the message prior to that. If that were all I was interested in doing I could stick it in a loop and call it done, but that’s not going to work. First, consider that station number. The way Modbus works, you can wire up a whole bunch of devices on the same port. Any message I send out will be received by all of them and they’ll look at that first byte of the message to decide if the message is addressed to that device. This is something that’s configured in the device itself and I can’t count on the device I want to send the message to always being station 1. Changing the station number changes the check digit so we need to be able to calculate this.

This is calculated according to an algorithm given in the documentation. It’s a 16 bit cyclic redundancy check algorithm. While Qt already provides a function for calculating the CRC of a QByteArray, it uses the wrong algorithm for the purpose here so I wrote my own function for that, building on my previous test program so that I could type in any arbitrary Modbus message (sans check digits) and the program would calculate and append the check digits, send the completed message out over the wire, and display both what it sent (so I can verify the CRC) and the response. Here’s the code. Input is a QByteArray with everything other than the check digits, output is a 16 bit unsigned integer that should be appended to the message before it is sent.

quint16 modbusCrc(QByteArray data)
{
    quint16 retval = 0xFFFF;
    int i = 0;
    while(i < data.size())
    {
        retval ^= 0x00FF & (quint16)data.at(i);
        for(int j = 0; j < 8; j++)
        {
            if(retval & 1)
            {
                retval = (retval >> 1) ^ 0xA001;
            }
            else
            {
                retval >>= 1;
            }
        }
        i++;
    }
    return retval;
}

We might also want to check that the replies sent back from a device were received correctly. If we take the whole response and pass it into this same function, the return value will be 0 if the message was correctly received.

I’m using an event driven system for reading data from the serial port and at only 9600 baud it’s not uncommon that my software will attempt to read a message before the whole thing has been received. When we send a message, we can know what to expect in returned data. The length of the reply depends on what we asked for. While we can calculate the response length we expect, in the case of an error that might not be what we get. Better to find out from the response itself how long the message is. So what does a response look like? The first byte identifies the device sending the response. It’s the station number from the message we sent. The second byte is a function code. We expect this to be the same as what we sent, but if there’s an error we’ll get a different value here. If there’s an error, we now know how long the message is. It’s five bytes with the remaining three bytes being what kind of error was reported and the CRC. Some functions have fixed sized replies. The ones that don’t use the third byte to indicate how many bytes of data follow. Once you have that number you can add 5 (3 for the data we’ve already read and 2 for the CRC) and now you know how long the reply should be.

There are a few things that still need to be done to get this integrated into Typica.

1) Connection configuration is slightly different from previously supported devices. I’ll want to provide a configuration widget that allows specifying the port settings. When I get around to looking at the documentation for other devices I might find that I need to expand on that a bit, but for now I intend to keep the options minimal for the device I’m testing against.

2) Internally Typica handles all of its temperature measurements in degrees Fahrenheit. Requesting the PV returns an integer value in whatever the currently selected unit is, ignoring the decimal point (so 500, 50.0, and 5.00 would all be shown as 500). Before getting measurements, the program will need to ask the device which unit it’s displaying its values in and where the decimal point goes so that any conversions that are needed can be performed appropriately.

3) Once measurements start, I could just have a thread loop, sending the request for PV out, getting the response back, and packing the measurement up to send to the rest of the program, but there are a couple issues with that. While we’ll generally want to read the PV as often as possible, we might sometimes want to do something like output a new SV so there needs to be a way to slip something like that in between PV requests. The other complication that should be considered early is that we may want to request the PV from more than one device on the same port. The program needs to give each device a turn. Neither of these is a hard problem. There’s a well understood solution, but it’s important to be aware of these things rather than have to rip the code apart later.

4) From there I’ll want to do some testing to make sure that code that’s currently working for a few minutes at a time and not moving much data over the port continues to work after a few hours of moving lots of data over the port.

No, you didn’t miss the day 2 update. We’re (that doesn’t include me) refinishing the floor at the shop so I left for the day before the transformer I discovered I needed on day 1 was delivered so I could take a nap before returning to help move everything off the floor and put drop cloths over the green coffee and equipment. I brought it home with me when that was done but didn’t do anything with it.

Today the goal was the same as day 1. Get the device powered up, reading measurements, and talking over RS485. The first two steps went very well. Wired the power terminals to a transformer, hooked up a thermocouple calibrator on the TC inputs, turned on the calibrator, plugged in the transformer. Measurements came up. I tweaked the settings a little bit (get a digit after the decimal point) and set various values throughout the normal range of roasting temperatures on the calibrator and found the device to be within 1 degree F of the value indicated. That’s very good.

RS485 communications was a little bit trickier. Depending on where you look the documentation for the adapter I was using indicated about four different things for how the wires should be connected and I made a bad assumption about how communications were set up on the device itself so that was a bit of a time sink that eventually had me trying to debug the connection with an oscilloscope. It didn’t help that when I tried to run Fuji’s software for setting up the controller I was greeted with this message:

Naturally I’m on x64. So, a little searching around and if an x64 version of this program exists I can’t find it. A peek at the manual shows me this gem:

So I’m guessing that the software was written 15-20 years ago and nobody has bothered to see if it still works on a computer that someone can still go to a store and buy. I wasted a lot of time with a trial version of some other software that I wasn’t able to figure out how to make work (I’m not going to name it because while its trial mode was really annoying the whole not working part could have been my fault). Eventually I gave up on using someone else’s program to test communications and hacked together a quick test program based on the communications manual for the device. I used QextSerialPort, which I’ve used to support a couple other devices in the 2.0 branch. It’s a nice set of MIT licensed (same license as Typica) classes that provides a cross platform way of using serial ports. The communications manual itself is really very good. Seriously, take a look at it. Hardware makers take note, this is how you should be writing your communications protocol documentation. Page 23 even provides an example of the main thing that Typica needs: how to read the PV from the device. I copied the example bytes from that into my program, got back a response that was the same as the example response aside from the last four bytes. No surprise there. The last two bytes are a check digit and the two bytes before that are the PV, which came back as 0x01F4. Convert that to decimal and you get 500. I had the calibrator set at 50 degrees and with one decimal place showing so this is exactly what I would expect.

So, I have a little bit of my own code now that can talk to the device and get back an answer. I hadn’t intended to write that until tomorrow, but it didn’t take very long once I started doing it and so far it seems to work.

Today I received two devices: a Fuji PXR-4 controller that speaks Modbus and an RS485 to USB adapter (it actually has both a terminal block for RS485 and a DB9 connector for RS232 and will let me use both of those at the same time, presenting two serial ports over 1 USB port). My goals for today were modest.

1) Hook the new controller to a coffee roaster. I brought home my SR1 electric sample roaster for this purpose not intending to roast any coffee on it but to use its thermocouple and power supply for testing purposes. Apply power, verify that I have measurements appearing on the panel.

2) Connect the new controller to the adapter. Plug it into my computer, install the software from the manufacturer and verify that it can communicate with the device.

There’s no sense in writing any code before these are done because I wouldn’t have any way of telling if something wasn’t working because the program was wrong or if it wasn’t working because the hardware wasn’t wired up correctly. This gets me a base line position of hardware works correctly and can communicate with the computer.

Well, I hit a snag on step 1. Before disconnecting the old display, I made sure to document all of its connections. The PXR-4 is half the width of the display the roaster came with so it wasn’t going to be a permanent installation (well, someone must make a plate that converts a 1/8 DIN cutout to 1/16 DIN right? I suppose I could have such a thing made, maybe even with a hole to mount a data port on the panel) but it’s not as if I bought it myself and beggars can’t be choosers. After doing that I went over the model number on the side of the PXR-4 to verify what exactly it was that I had and that’s when I discovered a problem. This model requires a 24VAC/VDC power supply, but the old display operated on 120VAC. It’s not going to work installed in that roaster and I’m not going to fry the part by trying that. A shame as the SR1 is a heavy machine and I could have saved myself some lifting. At least that cat was amused watching me bring it in.

So, plan B. Do I have any 24V power supplies that I’d be able to raid from something else? Nope. Lots of 12V, lots of 9V, but the closest I found to 24V was 19V. Ordered an appropriate transformer for delivery tomorrow and for a thermocouple I’ll just hook it up to my TC calibrator (or I could build a thermocouple, or hook it into the TC extension wires that are coming out of any of my roasters). That setup at least has the advantage of portability.

This part and competing devices are available in a 120VAC. Since my intent at this point was just to add support for continuously reading the PV, not having it in a coffee roaster is no big deal, but in the long term I think it makes sense to let Typica drive the SV. If the capability is already in the hardware, why not let the software take advantage of it? It’s not my first choice of how I’d approach automation, but if I can make it work that would be nice. If not, there are some other potential uses. Heck, even a manual SV control that lets you just type the value you want would be a nice usability improvement over the panel controls. I’ll have to see about getting at least that in for the initial feature set. When I have more time to work on automation, I’ll probably look into getting myself a wider unit in 120VAC.

For now, however, I think it’s time to beat up monsters in a video game. Later tonight I might see about back porting the framework that replaces (and substantially builds on) the DAQ class from the 2.0 development branch which allows for supporting multiple hardware drivers from a single build. In the interest of getting to the point that I can release a β test version quickly I’m not going to back port all of it (on the 2.0 branch I ripped out the Measurement class and replaced it with QVariantMap: a little more overhead in non-bottleneck parts of the code in exchange for a lot more expressiveness, but that means changes to everything that was previously producing or consuming those Measurement objects) but it would be nice if a single build for each platform supported all of the hardware. For the β release I’m unlikely to worry about anything other than Modbus over serial port devices, but it would be nice if version 1.4 had support at least for NI DAQmxBase devices (NI USB 9211, NI USB 9211A but possibly others as well), NI DAQmx devices (NI USB TC01, various 9211 based devices, possibly others, mostly Windows only with possible limited device support on Linux if you can figure out how to compile it yourself), Modbus RTU over serial port devices, and possibly also the Extech EX540 multimeter (there are a couple quirks that still need to be ironed out to get that working on the 1.x branch but even if I don’t have time for that I can apply a trivial hack that will work around the main issue) without needing separate builds for all of them on each platform. The other big thing I want to back port is a widget that hooks into that framework for configuring these devices. It probably won’t be as flexible as what I’ve been doing on the 2.0 branch (some of you want some pretty crazy configurations and I’m probably going to stick with the wait until 2.0 line for those) but that would let me drop down to only needing to maintain a single configuration (at which point I may as well just provide a setting for where that is, compile the default configuration in as a resource, and drop the configuration file prompt, one less step to using the program). Not sure how far I’ll get on that, but that’s what I’d like to do.

Remember a few posts back when I begged for some hardware so I could add Modbus RTU support to Typica? Well, someone came through on that and there’s currently a Fuji PXR device that speaks Modbus on its way and scheduled for delivery tomorrow. Once that support is in, it’s possible that Typica will work with or can be trivially modified to work with a lot of devices, some of which may already be in your coffee roaster (so you’d be looking at an RS485 to RS232 or USB adapter and a bit of wire to try Typica rather than buying more data acquisition hardware). While I had previously planned for the next version to be 2.0, I think this change is a good enough reason to put out a 1.4 release. Since this work will take some time away from my work on 2.0 and the release might slip a little more, I’m also going to see what I have time to back port from the 2.0 release and put a bit more polish into the profile translation and roaster calibration series features that I’ve recently written about and get those features out to other roasters a bit faster.

I thought some might be interested in finding out how quickly new hardware support can be added to Typica given access to hardware to test with and communications protocol documentation, so I’ll be sharing the progress on this as I go.

Hardware support is a tricky thing with Typica. Since I’m not charging for the program my development budget is approximately $0 so I can’t afford to buy all of the hardware that different people want to use with the software and I’m very uncomfortable with releasing code that I’m unable to test. I need to be selective. There’s only so much hardware that I can make regular use of in my shop and beyond that I’m considering a few things when deciding where to put my efforts.

1) Is this cross platform? I’ve done some work that’s Windows only, but I would much prefer to use hardware that doesn’t lock me into a platform, that I can use not only on Windows, but on computers running Mac and Linux operating systems as well.

2) Is there a publicly documented interface? If I can’t write code to talk to the device then I have no use for it. This is something that I’m going to be looking at before I buy the hardware (so an API that’s only documented on a disc that ships with the hardware is an API that I’m never going to see) and I’m not going to waste my time reverse engineering a communications protocol that could be silently changed any time a manufacturer decided to put out a new hardware revision.

3) Is it suitable for roasting applications? I’d rather work with hardware that produces measurements at an acceptable accuracy, without a lot of noise in the more significant bits of measurement, and at a fast enough rate.

4) How much does it cost? The cheapest devices out there have probably failed on more than one of the above points. Particularly expensive devices are difficult to justify.

For a long time Typica only worked with hardware from National Instruments. I’ve never had a problem with the devices I’ve received from them, but reports have been coming in for some time now of long delays for 9211 based devices, poor performance for the USB TC01 (devices not providing accurate measurements on arrival), outright incompetent customer service, and the price point for cross platform devices has gone up substantially from when I started. While I don’t intend to drop support for or stop using the NI hardware that I already have, it’s been clear for a while now that I need to provide some other options.

Recently I was asked about a particular device family. I’ve seen similar built into some roasters and this could communicate over RS485 using an old communications protocol called MODBUS RTU. There are RS485 to RS232 or USB adapters that would present this as a serial port on Mac/Windows/Linux. The communications protocol is documented and the device specific variations are also documented. It looks like the hardware is reasonably fast and accurate. I also really like the price point.

I got to thinking that if I end up writing code to speak MODBUS over a serial port anyway, I should try to make that sufficiently general that I could provide a way for people with other devices using this protocol to configure Typica to work with their devices. Before I start on this, I’d be interested in learning what devices that communicate with MODBUS RTU are already being used in coffee roasters. If you have a roaster with such a device or you manufacture roasters with such a device, drop me a note with the full model number of the device. If you have a link to communications documentation for the device, that would also be helpful but I’m willing to plug the model number into Google and try to find that myself. If you want to send me hardware, that would be awesome and deserving of special thanks in the About Typica window. I’d like to make this software usable for as many people as I can with hardware they’re already using.

Roaster heating. The blue line is measured values from the IR-1. The cyan line approximates what those measurements would be if I had coffee in the drum and were doing the roasting on the IR-12. The differences are minor at first, but significant by the end of even a light roast coffee.

Yesterday I wrote about the application of linear spline interpolation to calibrating measurements taken on different coffee roasters so that someone can develop a roast profile on one machine and match that profile on a different machine. Today I wrote the code that produces such a data series, tested it (it worked on the first try), and I just need to put a little more work in on that before I can really start using that feature. After that I’ll try developing three or four new roast profiles for upcoming lots of coffee and get a better sense of how that feature works in practice. I’ve also been getting spreadsheets from other roasters and it’s interesting to see what different people choose to track in the way of synthetic data (that is, data that is not measured directly but calculated based on recorded data). One of the things that I’d like to do in the design and development of Typica is not impose my own methods of roasting on others, but enable people to continue to work in the way they understand but presenting the information they want as they roast and in a way that makes the data easier to understand and use to make decisions in how they’re roasting coffee. This sort of synthetic data is something that I haven’t given enough attention to in the past. Some of the more obvious synthetic data sets include local first and second derivatives of the measurement series, differences between series (along both axes depending on what is represented by each data series), and temperature triggered process annotations.

What sorts of synthetic data series do you look at when trying to understand a roast?