CH340X DTR/TNOW with ESP8266/ESP32 BOOT and RESET

The CH340X is a small chip that does USB UART, and is part of a family of CH340 chips in various sizes. A UART can need few or many pins, and they do a really small one that is just Rx, Tx and RTS pins, which is the minimum you would want. But they also do a version specially aimed at the needs of small processors like the ESP to handle boot mode.

BOOT and RESET

Both ESP8266 and ESP32 (and many other processors) have a BOOT and RESET input, well, actually GPIO0 and EN.

You need these to program and debug. To program you have to set BOOT MODE, (which is GPIO0 pulled low) and pulse RESET (EN low and then high). For debug, you want no BOOT or RESET, or maybe RESET to restart the processor, but not BOOT mode (GPIO0 staying high).

What is weird is the various ESP based modules out there. I have seen complicated FET based links of RTS and DTR from a UART linked to GPIO0 and EN, and some with R/C combinations linked to RST and GPIO0. I think the logic here is for more traditional use of DTR and RTS, i.e. usually one has RTS active the whole time, so an R/C allows RTS to be activated (low) but EN to pulse low and back high. Similarly linking the use of DTR and RTS to allow a normal UART operation not to be BOOT MODE.

However, the esptool and idf.py as part of the ESP IDF, and even the tasmotizer tool and web page all treat DTR as BOOT MODE and RTS as RESET directly.

You can connect DTR directly to GPIO0 and RTS directly to EN, and it works perfectly!

What is even more weird is the number of modules you see with actual push buttons for BOOT and RESET on the board, which costs money to fit, even though they have a USB UART.

I can only assume some legacy designs and copying reference designs and so on.

All my designs connect DTR to GPIO0 and RST to EN, and work perfectly. Yay.

CH340X DTR

The CH340X has Tx and Rx and also CTS (input) and DTR/TNOW (output).

The TNOW is multi function though, the datasheet says :-

5.3. DTR and multi-mode MCU download

For CH340X, 6# Pin defaults to TNOW, weak pull-up during power-on or reset, output TNOW during normal operation used for half-duplex transceiver switchover. By connecting an external resistor with 6# pin, TNOW can be switch to DTR#, the two options are as follows:

1 If the 4.7KΩ pull-down resistor is connected to GND with the 6# pin, it will enter the open source DTR enhanced mode, and the 6# pin will automatically switch to the open source driven DTR# pin , which is used to connect the boot mode pin of MCU. The default DTR # is not output and is kept low by the external resistor, but the DTR# pin can be set to output high level or not output by the application program, which is used for multi-mode MCU download with DTR# default low level.

2 If a 4.7KΩ resistor is connected between the 6# pin and the 5# pin, it will enter the push-pull DTR enhanced mode, and the 6# pin will automatically switch to the push-pull driven DTR# for connecting to

the control pin of the MCU. The application program sets the DTR# pin to output high or low level for multi-mode MCU download with DTR# default high level.

This is excellent, and makes it clear they have considered the use of DTR as a BOOT MODE for processors. I have not tested the DTR to GND via 4k7 - I think they are saying it inverts, i.e. DTR drives high. I may be wrong.

But the second mode, DTR to CTS via 4k7, I have tested. Well, actually I misread / assumed 4k7 to 3.3V, and that does not work, but when 4k7 to CTS it works, and drives the pin as DTR allowing direct connect to GPIO0 yay

Why a 4k7?

But I am puzzled. I understand the 4k7 to GND, as it pulls down, and allows active drive high as needed. But why 4k7 to CTS.

CTS is an input, so the only use of the link is to allow the CH340 to confirm the connection from DTR/TNOW (output) to CTS (input). Indeed, it pulses that output low a few milliseconds after startup, presumably to test it is connected to CTS.

The thing that puzzles me is why the 4k7. With CTS as an input it is quite safe and valid to connect DTR directly to CTS, output to input. And, as you would expect, doing that causes the CH340X to switch in to DTR output mode.

I suppose one could argue that you may want to make use of CTS as well, and the 4k7 allows that even if DTR is outputting something different. Except that does not work. If you are using CTS, that means you have an active signal feeding CTS from somewhere. If that is the case, the test of DTR to CTS does not work (I tried it!) as CTS will not see the signal from DTR/TNOW via the 4k7. So the 4k7 does not allow that. Indeed, the only use may be if you have something driving CTS which does not drive it at power up - that would be a rather special arrangement and somewhat unlikely. That said, I suppose if CTS was a general purpose GPIO from a CPU which starts as floating/input, it could make sense and allow CTS to be signalled from the CPU.

So I'd expect the data sheet to say just link DTR to CTS and not use CTS, or use 4k7 if CTS is floating at power up and needs to be used.

Anyway I have done that (just linked DTR to CTS), and it works nicely.

USB-A tracks to CH340X

There is a snag though - using DTR to drive GPIO0 works, but on my Shelly board it is drive GPIO0 that is connected internally in the board, with some pull something, and an LED, and so on. So the CH340 might "see" the pull down on it when it starts perhaps. So I have to drive through a buffer in that case. Thankfully that is basically the only case, as normally it just goes directly to GPIO0 on the connected ESP module.

ESP8266 vs ESP32

When I started using the Espressif processors I started with the ESP8266 and Arduino development platform. It is all C++ based (which is not really my favourite). But it is a really good starting point for anyone wanting to play around with microcontrollers.

One of the key things here is the available hardware. A "programming" lead is just a USB serial which is available for a few pounds, The Arduino development platform is free to download. But most importantly there are thousands of very cheap ESP8266 based devices on the market, easy to use, and many easy to access programming pins. The ESP8266 is possibly the most popular basis for IoT (Internet of Things) devices like smart light switches and the like.

However, having managed to make some PCBs (milled from copper boards) for ESP8266 based modules, I eventually decided to move on to using the ESP32. It had a number of advantages as a processor itself, but also the ESP IDF which works on linux and MacOS making it easy for me to code in C, which I prefer. The ESP IDF and various libraries are still a work in progress and far from perfect, but generally work well. I have made a load of projects based on ESP32.

But being ESP32 means I lack the thousands of ready made modules that I can use. I have designed and made loads of circuit boards myself, including a complete access control and alarm system.

What is now nice to see if that there are finally some ESP32 IoT devices coming out. Manufacturers are moving to ESP32 based boards. One of the most useful I have found is the Shelly 1. What is especially nice is it has direct pins for programming access - without even taking apart. It is simple, just a switch input and a relay output, but runs of mains, or 12V DC (do not use the GPIO pins when running on mains as they are live).

Shelly have just launched the Shelly plus 1, which is basically the same as the ESP8266 based Shelly 1 device, but ESP32 based. It also has programming pins easily accessible (though smaller), a relay and switch input. In this case the 12V is a screw terminal rather than a link you have to change.

The good news though is I can load my own code, ESP IDF based, on to this. It means I can use these modules directly on, say, my access control and alarm system. I had found I needed various simple 12V single input modules for the likes of a door contact or fire alarm input and previous used boards I made. But at this low price the Shelly plus 1 is a no brainer, as they say. Using my code I can set secure boot, and encrypted communications as part of the alarm system. Even something simple like an alarm bell could use one (single output for bell and single input for tamper).

My first actual project with one of these is for a servant bell system. I have an actual bell, which I want to "ding". I am using 12V DC, but the bell really only "rings" using AC. Using DC you need a small pulse to do a clean "ding". Too short and the clanger does not hit the bell. Too long and it hits it and holds, making more of a "clunk" than a "ding. Get it right and you get a nice clean "ding".

I was using tasmota32 code for this, but that is designed to do pulses of 200ms at least and not that consistent - perfect for blinking lights and the like, but no good for what I need.

So, back to my own code, ESP IDF based, using my existing libraries, and I am now starting something a bit like my own tasmota in a way - programmable inputs and outputs. Expect the outputs (which is all I have coded so far) are configurable to the millisecond.

I have found 15ms seems to be the sweet spot to make the bell ding...

UK smart light switches

Obviously I have tinkered with smart light switches, and the like, for a long time.

You only have to look on Amazon to see a lot of different options now, but there was one simple combination that has been lacking. I have been looking for this for ages, and finally they now exists - the Shelly 1. But more on that at the end of this blog post. First a bit of background.

UK lighting circuits

One of the problems is UK lighting circuits have a live and switched live to the light switch from the light fitting. This means that, at the light switch, there is no neutral. The only way to power a "smart switch" is through the light bulb, which is not ideal. Indeed, it is not that reliable as it also depends a lot on the type of light bulb in use. This means that almost all smart light switches need the wiring changed to allow live and neutral to the light switch. In simple partition walls that is not too hard for an electrician to pull through a new cable, but it is not always so easy. The other big issue is that you almost always have to change the back box from a shallow back box to a deep one, and that can mean chiselling out some brick, etc.

Basically, fitting a smart light switch in the UK is a pain, needs an electrician to do properly (and I am not even trying to address the regulatory issues here), and costs a lot more than just the switch itself.

There are now, finally, some UK variety smart switches, e.g. DS-102L. It has a bypass capacitor to fit across the light bulb so as to allow enough in-line power for the switch. This again is fiddly for a non technical person, but saves running new wiring. I am making the huge assumption that this is valid wiring in the UK as well.

Some options

There are a few approaches to smart lights though...

• A smart switch. But as mentioned, in the UK, this is not always easy.
• A smart bulb. Simple, but tends to cost more (I'll not go in to these here, but Shelly do one!).
• A smart in-line power relay in the lighting circuit.

Yet more problems

The other problem is how you control your smart light switch. Obviously some sort of app on your phone, or integration with Alexa or Siri or something. But then what?

We tried - we really did. I got a simple smart light switch, and we decided to try and get it working as intended. I have used these for years, but always re-flashed with new firmware, so trying it as sold was new for me. A (non technical) relative wanted to simply be able to work some lights, from her phone, remotely, so this seemed like a good idea. I also asked my son to try, as a new pair of eyes. No joy!

We even set up a bog standard router with NAT and WiFi (yes, NAT is evil), and still could not get the damn thing to work as intended. They come with an app, which was easy to download, but then we could not get anything to actually work. This did surprise me, to be honest.

We ended up putting a raspberry pi running MQTT (mosquitto) and a tasmota flashed in-line power plug for a lamp.

Using the Internet

The other concern I have is that these things normally use servers somewhere in the Cloud, i.e.on the Internet. No idea where. No contract in place with whoever runs the servers. No recourse if they stop working. No idea if usage patterns are logged or sold somehow. And all this relying on working Internet.

Call me old fashioned, but I like a light switch that does not rely on working Internet and someone else's server in China!

Of course this means having my own "hub" of some sort - in my case a raspberry pi and MQTT server, but this could be some "home hub" or some such.

Tasmota

If you have not encountered Tasmota before, do take a look. It is free open source software that runs on most of these smart switches and devices. It means re-flashing the device, and they vary in complexity from "having a header you can just plug in to with a serial lead", to "soldering bits of wire inside the switch". But once re-flashed they can be re-configured and re-programmed over the air (WiFi). As for actually flashing the code - you just need a simple serial lead and the tasmotizer app, and it is very easy.

The key thing about Tasmota is that it works using a simple standard called MQTT. Having an MQTT server in your home is cheap and easy, and can integrate with various home automation systems. It does not, then, rely on any Internet connection or third party servers. It can work with various home hub / automation systems as well.

Some options

The main three options I would consider are these. All can be flashed with Tasmota.

• Sonoff basic: This is a neat, very cheap, in-line, 10A switch. It has live/neutral one end, and switched live / neutral the other. It has a tiny button and LED as well, which are not that useful. The main downside I have found is they have a tendency to die after a while.
  
  
  
• DS-102 light switch: There are a lot of these type of smart light switches. The key thing with this particular model is that it has actual tactile micro switches behind the buttons. There are many that are capacitive, so harder to work in the dark when not looking. Some also have LEDs you cannot control. But the DS-102 seems about the best I have found so far. They come in one, two, and three gang, and as mentioned they have an option for live only, supplied with a capacitor to fit across the light fitting. One big down side is you have to solder wires to re-flash them (though people have made rigs with pogo-pins to do this).
  
  
  
• Shelly 1 relay: This is what I have been waiting for! It is a relay, like the sonoff basic. It costs more than the sonoff basic, but typically less than one of the smart light switches. See below.
  
  
  

The Shelly 1

I have been after this for a long time - I even considered making one myself - something simple like the Sonoff Basic, but with a proper switch input. The Shelly 1 seems to be ideal for light controls in the UK.

It can go behind the light fitting in the ceiling where the live and neutral are already present, so solving the fact we don't have neutral at our light switches. But it can take the switched live as an input. Obviously it can have a default mode of operation where the switch works the light, so even no Internet or WiFi does not stop it working in the obvious way.

It has a simple header for re-flashing (please do take note of the warnings about these being at live voltages when connected to power!). So loading Tasmota is a doddle.

This means that with no extra wiring, and no changing to a deep back box, and no change of light switch to something cheap and tacky white plastic. It is easy to make a normal light fitting WiFi connected.

It is also 16A not 10A, so much more useful than the Sonoff basic (obviously for things other than lights, which get nowhere near that rating).

I have one coming today :-)

P.S. For those that have not used tasmota, you can have an input as a button or a switch. A button works toggling an output on pressing the button. A switch toggles the output on change of input. So a normal light switch would be connected in switch mode, and turn on/off when changed. If you override via MQTT you simply swap the way it works, much like a normal two-way light switch arrangement, so the switch now works in reverse until you next override.

P.P.S. Smaller than I expected - this is really neat, and can even run off 12V or 24V-60V if you need. This is exactly what I wanted!

P.P.P.S. I had not heard of Shelly before, but they have loads of cool stuff, all looks quite sane, including the simple to re-flash header, and the fact they will do MQTT anyway. Very cool. shellystore.co.uk

(Do always follow local electrical safety regulations, please)

Moving to ESP32

Someone already told me that once I move to ESP32 I won't want to go back to ESP8266.

Well, apart from the fact the ESP8266 is available in a tiny ESP-01 package, I think they may be right. The ESP32 in a WROOM-32 package is slightly wider than the ESP12 package, but because the GPIO pins can be mapped any way you want, pretty much, the tracking takes less space. So overall I was able to make an equivalent board in less space using ESP32.

The big challenge for me with ESP32 was the concern over the fine pitch - 0.9mm contacts at 1.27mm spacing. I was not sure I could either mill a PCB or solder a board at that pitch reliably.

I was wrong, I can!

New door controller using ESP32

Fixing the X axis backlash on my mill was a big help as well. So yes, I can use the ESP32 modules.

The other concern was the price, but it seems that getting a proper ESP32 is as cheap, if not cheaper, than an ESP12S. Yes, some cheap ESP12S can be got, without CE marks, and missing FCC ID, and not quite working well on WiFi. But £2.91 for ESP32-WROOM-32 from a reputable supplier is pretty good.

There are advantages to the ESP32, obviously, including much more RAM, faster, dual processor, more GPIO, more UARTs, and so on. It is a good comprehensive IoT device.

The other big change I want to make is moving away from Arduino IDE. Yes, Arduino makes things easy, but it turns out the Espressif IDF for ESP32 is pretty good. It has the key things like https, MQTT, OTA, and all sorts as standard.

So my plan is to make an ESP32 IDF based version of my door controller, including PN532, VL53L0X, etc. This should not be too hard, and a lot of my existing code can easily be ported to a simple C code environment.

I am, however, impressed with the documentation - not only of the APIs in the IDF, but also the technical reference manual. It actually details all of the registers clearly.

The other impressive thing is Espressif seem to to be trying to encourage some best practice in IoT. They have secure boot and signed images as a standard feature. Even https libraries, as standard. It means that making a reasonably secure IoT device is easy. Well done.

I managed to get the build environment working on Mac and linux with no problems, following the clear instructions. I built the "hello world" example, and flashed it with no problem.

I am off on a 4 hour flight shortly, so I have the 669 page PDF of the technical reference manual on an iPad for some light reading on the flight.

Of course, once I have this off the ground, I can look at Bluetooth, and CAN bus, and all sorts of new fun things.

Environmental sensors

For my next little project I am thinking of making a good little environmental sensor box, mainly for CO₂ and temperature as they are the two things I have some control over.

ESP8266 based, why not. But I have been looking at the sensors you can get, and bought a few.

Are all CO₂ sensors crap?

Well, I tried a few, including this CJMCU module (CCS811 based) and this AppliedSensor module.

Sorry to say, but both are crap! And the detail is in the small print - they estimate CO₂ from other sensors, and are very easily confused. I have some isopropyl alcohol and you use that anywhere near them and they quickly say 18000ppm CO₂. Even without such factors they swing wildly all over the place.

It seems you need an NDIR based sensor, and I found this one which seems to actually work! This SCD30 sensor seems to be the business, with CO₂, temperature and humidity (floating point!) via I²C.


Next step - sort a suitable display and box.

I can then use the temperature to control the air-conditioning, and the CO₂ to control extractor fan speed.

Door control

My door control and alarm system has made some progress.

The main change is making access control a bit more distinct from alarm system. The doors at the office were "falling off" the WiFi occasionally, even if only for a few seconds at a time, a few times a day.

I suspect that can be fixed, and it makes a huge difference what channels we use. The AP report something like 60 interfering APs.

So the change is to make the doors work autonomously which means the door controller has to be able to authenticate the fob/card used but also know enough to tell if it is allowed access.

I have actually created six levels of integration and autonomy for door control linking to alarm system.

This means things like times of day, depending on day of week, which doors they can access, and expiry dates and the like (i.e. some way to be sure a lost card is now invalid).

However, the DESFire cards are a challenge at best. I cannot have any operation that produces an error as that kills the authentication. So I have to only read files that exist, and only read the right length of the file, etc.

My original design was to have files for various aspects of security. Allowed doors; Barred doors; Start and end times; Start date/time; expiry date/time. The idea wast to get the file ID list and decide what to check depending on what files existing.

But even that is a challenge if any files can be variable length. The allowed/barred lists are like this, as are other files. So you have to check the file size for each file first.

Before you know it you are doing dozens of operations to the card, each taking tens of milliseconds.

So I ended up making a system that uses a single "access file", which I expect to exist. I ask the size and then read it. In that file I encode the various things I need like allowed and barred doors, and access times. But it is just those two operations on the card and hence a quick operation.

The end result is an autonomous door control system on an ESP8266 based door controller. It even has expiry update so that a card not used for X days expires. I obviously have blacklisting, but that also zaps the file if a blacklisted card is used.

The next step is some key roll-over logic as well as extra belt-and-braces old card expiry logic.

This is giving a fast enough process for door control. Yay!

Soft RS485 UART using interrupts on ESP8266

My non technical readers can skip this :-)

Background

As you will have noticed, I have made a complete access control and alarm system based on a Raspberry Pi but working over RS485 bus connections to standard Honeywell Galaxy parts like RIO, Max Readers, and Keypads.

As you will have noticed, I have now started making WiFi connected parts for the alarm system based on ESP8266 parts (ESP-01, ESP-12F, ESP-12S). These include a small ESP-01 based board with RS485 driver that fits inside a Galaxy keypad talking RS485 to the keypad.

As you may have noticed, I have NFC readers sorted and working with secure NXP MIFARE DESFire EV1 cards. These are way more secure than poxy proxy cards used by Max Readers.

Challenge

The concern some people have raised is the reliance on WiFi. It will do the job, but it is not that hard to disrupt externally. Now, much like using a screwdriver on a Max Reader, it will set off the alarm to disrupt it, but it is perhaps a tad easier. One has to consider if it is easier than simply forcing the door or breaking a window, which is always the benchmark for any security :-) But it is a concern.

So can I make the door controller, which does the secure cards, look like a Max Reader and do RS485 to a conventional Galaxy system? I think I can...

Approach

Initially, for the keypad system, I used the built in serial drivers on the ESP8266. These are good, interrupt driven, well supported, except...

The code on the ESP8266 using Arduino is the NONOS SDK, and means one thread for all. It can be delayed a bit for all sorts of reasons, even many milliseconds. For serial that is all handled behind the scenes on interrupts this is no problem, until is it RS485!

The problem is that for RS485 you have to enable and disable the line driver around your transmission. Initially I simply set a time to disable after poking the message in the Tx queue. But that could be delayed and mean after transmitting we held the line driven for too long.

The solution was to wait in the main loop task until sent, and then disable the driver. This is not nice, as any other operations of the device are delayed, but for the keypad driver application there is nothing else. So it works well.

This will not work for the card reader as it has to, well, talk to the card reader, and inputs, and outputs for the door control, and so on. Also, if I am a slave device (unlike the keypad where I am master) I am expected to answer polls promptly and not hang around for 10ms randomly.

My plan - interrupts! Make a soft serial RS485 UART on an ESP8266 to do the grunt in the background.

UARTs

A UART (Universal Asynchronous Receive/Transmit) is a device to send and receive old fashioned serial byte / character data. The concept dates back to what? the 50s or 60s. It is simple...

Each byte, or character, to be sent is coded as a series of 1 or 0 on a bus. What 1 and 0 mean in terms of current or voltage depends on the flavour of system (RS232, RS485, RS422, etc). Each bit time (1/Baud rate) the line has a 0 or a 1. The line is idle (1) and then the byte/character starts with a start bit (0), bits in the byte (low bit first), then any parity bit, and then stop bit (or bits) which are 1.

Obviously both ends have to agree the rate (Baud rate) for bits, and byte size, and parity present and if even or odd, and even number of stop bits.

The Galaxy uses 9600 Baud (so a bit every 104⅙µs), 8 bit data, no parity and one stop bit. Simples!

Sending is easy, you need some sort of timer, and you set 1 or 0 on the output to do the start bit, data bits, and stop bit. The Arduino ESP8266 SoftwareSerial sends in real time from the command using careful timers, it seems. I plan to use interrupts.

Receiving is a tad harder. You have to find the start bit for each byte, and then sample each bit, ideally in the middle. But once you have the bits you shift in the data bits and check the stop bit is 1 (else an error). Not that hard?

Edge interrupt on rx?

Interestingly the SoftwareSerial code simply interrupts on each edge and looks at time from last edge to work out the received data (I think). That is one approach but I decided to clock in bits on interrupt - I just need to sample mid bit somehow.

My initial approach was to set an interrupt on the falling edge of the data line to set the recurring timer half a bit later. However, this has all sorts of challenges, no least of which is race conditions on a regular per-bit interrupt and the edge interrupt. I did work, but not 100%. I am sure I could have solved the race conditions, but actually there is a good reason to stick to simple regular interrupts.

So the approach I took was just a regular interrupt, and this was 3 times per bit. This means ⅔ of the interrupts just count a sub bit and return in most cases, so very light weight in terms of time. But what it does mean is that I could in fact have multiple instances of RS485 busses running on this one interrupt if all the same Baud rate. I have not yet coded that though. It is, however, nice and simple and working for one instance well.

Why 3?

The 3 times per bit is simple. For Tx it makes no odds, skip 2 out of 3 interrupts and set the line to 1 or 0 for the next bit to send. For Rx, it is the same for sampling, except when waiting for start bit. If waiting for the start bit you check for the line going 0 every interrupt, and if it does set the sub bit counter to sample one interrupt later and then every 3 interrupts.

This is not as clean as an edge interrupt adjusting the timing to the microsecond, but it does mean that we can have sampling of the centre of the bit, ±⅙ of a bit which is more than adequate. Doing 2 times bit rate is no good as you could be sampling in the middle or right at the end. Hardware UARTs usually do 16x bit rates, but 3x is fine in my opinion.

This is how I have coded soft UARTs for over 20 years dating back to PIC16C84s, realy!

Messages

The way the RS485 bus works on Galaxy systems is that messages are sent as a sequence of bytes with no gaps, and then a gap - no length indicator in the message (but there is a checksum). So I coded the system to handle this, and even, if we are a slave, auto respond with current status response.

This allows the application to work on a message level, and if it is slow, we answer polls promptly anyway with current status. The driver also does the checksum.

Interrupts on ESP8266

Now the challenge is making it work on an ESP8266. They are rather special. They are a weird mix of old and new. The ESP-12S has 4M of flash! As an embedded controller that is huge! But it has around 80k of RAM (yes, k) and I had a BBC micro with more in the 80s (32k main and 64k second processor). The fact it does TCP at all is mental, but it does, and it does WiFi too.

The usual pressures on coding are complex - embedded systems have pressures on code space, RAM, execution time, power, all sorts.

In this case interrupts are special. The processor has a cache for the code, but this cannot be used by an ISR (presumably some interrupt process handles cache fails). This means the ISR has to be in RAM.

The trick is marking the code you use for the ISR with ICACHE_RAM_ATTR attribute. The RAM used is small, so not only does the ISR have to be fast it has to be small in code space too.

Thankfully a UART is simple code in terms of space and time. Small TARDIS code, even?

This worked, well, sort of, it worked mostly if started after about a second after boot. It turns out that using digitalRead, digitalWrite and pinMode is bad as these Arduino functions are not marked ICACHE_RAM_ATTR. Instead you need to find the underlying GPIO_REG_WRITE and GPIO_REG_READ functions. That then works reliably. The GPIO access is also faster, which is good, but very processor specific.

The result is working RS485, on interrupt, on an ESP8266!

I even added a clock track (blue on that trace) for debug - toggling an output for tx and rx clocking per bit.

It works!

This was deployed as new code to talk to the keypad, but means I can also act as a slave and pretend to be a Max Reader and provide a fob ID derived from the NFC card ID when checked using AES handshake.

It is also quite important as I am using the hardware serial at 115200 Baud to talk to the PN532 NFC card reader!

P.S. I have tested now as both master talking to Keypad, and as slave, pretending to be a Max Reader.

Choosing interface for PN532 (I²C, SPI or HSU) to ESP8266

I have been working with ESP8266 controllers and NFC readers. My preferred reader is the PN532 based Elechouse reader. As I have noted before, buy from Elechouse direct as there are a lot of bad copies!

However, when connecting to the processor I have a choice of interface. It is one of the rather nice features of the PN532. There are three interface types I²C, SPI or HSU.

So how to choose...

I²C

I²C stands for Inter-Integrated Circuit, so the acronym is IIC, but someone thought it would be fun to treat letters in an acronym in the same way as algebraic variables and write II as I². Arrg!

It is a simple bus for connecting devices and uses a clock and data line. Wikipedia has details (here). Both clock and data lines have a passive pull-up resistor and are actively driven low by master or slave. The protocol is well defined, and works well. It allows multiple devices on one bus and is very popular for a lot of peripherals.

Looking at the Elechouse web site I saw this: "On-board level shifter, Standard 5V TTL for I2C and UART, 3.3V TTL SPI" which rather concerned me. The ESP8266 is all 3.3V, and I did not want to have to mess about. This immediately put me off using I²C. However, looking at the schematic, the level shifter is completely optional, in that the same two pins are used for SPI, I²C and HSU and connections have these at the 3.3V level direct from the PN532 (via 100Ω) as well as via the level shifter, so it can be used for HSU and I²C if you want.

✓ Only two wires, simplifies wiring and frees up GPIO pins
✘ ESP8266 does not have hardware I²C support, so uses more processor and is slower
✓ ESP8266 has software I²C which means almost any GPIO pins can be used for clock and data, making PCB layout much simpler.
✘ Data sheet suggested level shifted to 5V (which turns out to be optional)
✘ Both clock and data use passive pull up resistors, meaning they can be subject to more interference on long lines

SPI

SPI stands for Serial Peripheral Interface. Wikipedia has details (here). It is more complex than I²C in the way it works - using 4 wires not 2, and allowing different configurations. It is also quite fast, allowing several MHz operation.

The ESP8266 has hardware support for SPI, which is good, but the downside of that is that specific GPIO pins are used for 3 of the 4 signals, so PCB layout is more complex.

✘ Four wires, so more to connect from controller to reader and uses more GPIO pins

✓ ESP8266 has hardware support, so allows faster working

✘ ESP8266 has hardware support, so the pin assignments are fixed making PCB layout more complex

✓ Lines are driven (from master or from slave) rather than using passive pull ups

✓ 3.3V direct connection to ESP8266

HSU

HSU stands for High Speed UART. This is simple old fashioned serial communications, on a Tx and Rx line, and defaults to 115,200 Baud.

The ESP8266 has hardware support for UART, and will work at these speeds. The ESP8266 library has good, interrupt based, serial handling. Unlike SPI there is actually a choice of pins that can be used, but this is still a tad limited. Of course the ESP8266 uses the UART for other things like programming and debug, but there is an additional Tx available for debug (Serial1).

✓ Only two wires, simplifies wiring and frees up GPIO pins

✓ ESP8266 has good hardware and library support for serial UART

✘ Pin usage is a fixed, but a choice of two sets

✘ Data sheet suggested level shifted to 5V (which turns out to be optional)
✘ Serial on ESP8266 is used for programming and debug
✓ Tx and Rx are driven (one each way) rather than passive pull up resistors
✓ The same connector can use used for programming and connection to PN532, reducing number of headers needed on the board

What did I do?

I was initially put off by the level convertor comment, leaving SPI as the only option. The first project I made using an NFC reader was connected to weighing scales, and used the serial on the ESP8266 to talk to the scales, so that rules out HSU. So SPI it is.

There are 4 connections needed (so 6 wires including power and ground). This means you can use 5 pins in a row on the ESP8266 (VCC, D13, D12, D14, D16) and GND to make a 6 way connector. Sadly the pins are not quite the same order as on the Elechouse PN532 board. This just means the connectors need wiring carefully.

This works well. I then used the same design for the door controller.

Hindsight

Firstly, the fact that the level convertors are optional means I²C and HSU are an option. I could have, for example, used I²C on a smaller ESP-01 for the scales rather than using an ESP-12F and SPI. However, the use of passive pull-ups is still a concern if the reader is on any length of wire from the controller as may be the case on a door controller. I could have used HSU, as the ESP8266 libraries have a soft serial mode, which can work at the scales 9,600 Baud rate, so could have used Tx/Rx for PN532 and one of the other pins to talk to the scales serial interface, and used an ESP-01.

I have since tried HSU, which just works, and is only 4 wires to the reader (power, ground, Tx and Rx). I was concerned that it would be slower (115,200 Baud rather than 2MHz) but actually it is slightly quicker (28ms rather than 30ms for InListPassiveTarget on same card), and a lot of the time is the PN532 operation and the NFC card itself. The Baud rate can also be configured!

It also turns out that had I used the more usual GPIO15 as the SPI SS pin not GPIO16 I would have actually wired the PN532 such that it could be used for HSU or SPI, as the PN532 uses MOSI for Tx and SS for Rx and the ESP8266 alternative Tx/Rx for UART0 are on those pins!

In practice, what I have done is make a 4 pin header with GND, 3V3, Tx and Rx. This is the same order as the 4 pins on the Elechouse board, but is also a header that can be used for programming the ESP8266.

I am slightly kicking myself for not trying HSU and I²C to start with, but to be honest I was just pleased to get things working. Freeing up two extra GPIO pins is nice, as is having fewer wires to reader, and pins in the same order both ends of the cable. The libraries do, however, make it very easy to change interface. Just remember to set the switches on the Elechouse board to match what you are using.

Tipping the scales

I am getting really good at upgrading the scales to do WiFi now, and read cards. The card of choice is a Monzo card which works perfectly well as an ID for weighing yourself. Notably Amex have an ID of 00000000 which is useless.

Really neat PCB, locking molex connectors, cables, the lot. It is working well. Marsden kindly sent the service manual so I can set to "kg" and "st/lb" modes for those that like old-school weights.

The system has a QR code on the scales, which allows you to register any card against your email address. You then weigh yourself and tap the card on the scales to record the weight.

When you tap the card it beeps to confirm it is working, and keep beeping until a stable weight is recorded over WiFi to the server. Simple UI if ever I made one!

Sadly, I have fried a couple of boards whilst doing these upgrades, but Marsden do spare parts (not cheap) and whilst I have a few clues what I may have done, I am not 100% sure of how I killed them. I think, with care, and anti-static, and being neat (avoiding any shorts, or diodes backwards) I can do this reliably now. Obviously very unofficial so I cannot expect any help from Marsden. They do make very nice scales though.

The next challenge is to make the web site that holds the data have graphs and sharing options and so on. At present it is just a list of weights. But even that is working for relatives using this solution.

This is turning in to the new fat-shamers club or something, not sure.

But what was the first thing on the web site? after registering the domain? well, it was getting an excellent lawyer to put together the privacy policy for us and ensuring we are GDPR compliant.

You have to love any lawyer that covers standing on scales with your pet cat as part of the privacy policy. Thank you Neil.

Relays

In doing a door entry / alarm system, one thing you need is outputs that control things, like a door lock. Whilst we have nice motorised locks for external doors, we also have mag locks for internal doors, but either way you need 12V and some reasonable current (around 0.5A). You cannot do that from a 3.3V output pin that can sink 20mA max.

So you use a relay, obviously.

Mk1: ILS relay board

First off, I found a nice, and reasonably small relay board. It works off 12V and designed to take an ESP-01. It even comes with one.

All I need to do is add my own firmware and I have a 12V powered relay on the WiFi. Simple.

There are a lot of advantages, obviously :-

• Simple single complete module for relay control
• Comes with ESP-01
• Runs off the voltage we are using (be careful, there are 5V versions too)
• No wiring, no PCBs, just load my own firmware
• Also available in 2 and 4 relay modules

 

Mk2: More connectors

Unfortunately I don't just want a relay output, I want a door control. So I made a PCB that allowed an ESP-12F and hence more connectors. With some careful design I made it fit on top of the relay board!

This works. But is starting to get quite bulky. Bear in mind these usually have to fit in the back of a electrical outlet box, e.g. with an "Exit" button or similar. I was also not happy at how warm the main relay board got - it uses a simple linear regulator.

Mk3: Separate 3.3V relay

Time to try using the nice Pololu regulator boards. With a regulator I don't need to use the 12V relay board, I can have a stand alone board. But I still need a relay. So time to use an off the shelf 3.3V relay.

Mk4: Solid state relay

That worked, but really it is still rather chunky. After a bit of hunting around I found some nice solid state relays by Omron. I have always seen solid state relays as quite chunky, but it seems they can come in quite a small package, more than adequate for 12V at a few amps even...

The result, as you see, is a relatively small PCB, with regulator, and header to connect to the NFC reader (it does just fit passed the regulator, honest), and contacts for 12V in, exit button, door closed reed, and lock engaged contacts, as well as a relay output.

Now I just need the sort the right NFC reader and change over one of my doors :-)

Mk5: A remote I/O board (RIO)

I've got the power!

In messing with bits for a door entry / alarm system, everything is currently powered with 12V. However, these little ESP8266 processors are 3.3V.

Obviously I need a regulator, and as I am making these up using a milling machine, a small self contained module I can solder on to a PCB as a single component is the obvious choice. Back to Amazon, and I find something simple and cheap :-

It has three 0.1" pitch leads, and so easily mounts on the PCB. It has an offset with a header as it has components both sides. Works well though.

However, I noticed it gets a bit warm. Not really a huge surprise, but even when just powering from 5V in my Galaxy keypad design it still is a touch warm.

So I did a bit more research and found these tiny switch mode supplies, Pololu D24V5F3. A few pounds more expensive, also available from Amazon, but really really nice:-

• Still a very small design 0.4" by 0.5" (yes, bloody inches!).
• Components only one side, so can mount flush to my PCB making easy for something like an ESP-01 to fly over it and save more space.
• Cool. I mean really cool. They say 2mA, and given that my ESP-12F plus RC522, running off 12V on one of these is 30mA total, I am impressed. I was seeing 140mA previously.
• It also has a shutdown input - which I suspect I could use with a button on a battery powered device so you can power up and then have s/w power down. I wonder if I can even do something clever with sleep mode on an ESP8266 shutting down its own power while it sleeps - may need a capacitor somewhere in that :-)
• Will accept from around 3.5V up to 36V as input!
• They do other voltages, and current ratings as well, and step-up regulators.

Actually a D24V6F3, the D24V5F3 are on order

Basically, I am working through the total maze of too much information trying to weed out the good from the bad and work out what sort of components are good. I think this comes in to the "good" category for voltage regulator.

New life for old tech (Galaxy keypad)

I wanted to convert a Honeywell Galaxy alarm panel keypad to be on WiFi.

Simples :-)

Start with a keypad :-

Open it up...

There is not a lot of space...

(This had a proxy reader, which I removed)

We'll need a few parts, including a freshly milled PCB. All from GitHub.

No space for the spacer...

Fits nicely...

Not forgetting a wire link - sadly I could not manage a layout without this...

Add the RS485 convertor and power regulator and some wires, crop the leads nicely...

Looks tidy... (I then sprayed the bare copper with clear lacquer, by the way).

They keypad has handy 5V pads...

And the A/B wires...

And it fits in nicely...

Put back together...

And it works!

WiFi for an alarm system?

I'll post more details later I am sure, but I have again been working on my "Solar System" door entry and alarm system.

The existing system works using Galaxy RS485 peripherals like a "Max Reader". This works, but is there a better way.

Well, WiFi, maybe...

This is a tricky matter to consider. We know WiFi is not as "contained" as wires, meaning more vulnerable to attack and disruption, to say the least.

So, let's consider some pros and cons.

RS485

• Generally very reliable as wires usually are - that said, it has limitations - notably number of devices on the bus (both in terms of physical aspects of RS485, and polling time constraints).
• Hard to access if in the walls, inside the building, but things like Max Readers expose the bus externally (albeit at cost of tripping tamper).
• With any access (e.g. Max Reader) very easy to disrupt (albeit with tripping tamper).
• Very insecure - I have done a proof of concept with access to the bus that can inject messages to open doors, etc, without the slightest hint of a tamper alert from the Galaxy panel.
• Slow
• Needs wiring back to the panel.
• Vulnerable to snooping if you get physical access to bus
• Hard to do redundancy

WiFi

• Vulnerable to RF level attacks (albeit tripping tamper)
• Vulnerable to protocol level disruption (if you get in to WiFi) such as WiFi disconnects or IP flooding or TSP RST flooding.
• Secure if using TLS. Cannot be snooped or faked.
• Fast
• No need to wire back to the panel
• Easy to add a lot of devices (subject to access points that allow lots of associations)
• Harder to do battery backup, but possible - need 12V access points - can be done.
• Possible to do multiple AP fallback and redundancy

So I am making my SolarSystem GitHub project handle WiFi connected devices as an option.

• ESP8266 code for modules
• PCB layouts for modules
• 3D printed cases for modules
• TLS MQTT over WiFi with certificate pinning and checking
• SolarSystem code allows mix and match of RS485 and WiFi connected modules

Not finished yet, but most of the device code in place. I even have a way to use a Galaxy keypad on WiFi!

The door control logic is a relay board with NFC reader, and I am working on proper AES secured cards. The 125kHz proxy cards are stupidly easy to copy!

PCB milling

My first attempt as the RFID reader version of my "scales" system meant a lot of enamelled wire...

It was a tad messy, and time consuming. But worked well.

I decided a small PCB would be a better solution, and the answer for one-off PCBs is, of course, a milling machine.

Nearly 30 years ago I used such a machine when working at Nokia. It was very useful, and expensive. These days you can get a small CNC machine for under £200!

I purchased one from Amazon (duh!).

It took some assembly, to say the least, but plenty of good videos on line.

It has an arduino, which allows moving the head and running code (.nc) files from a micro SD card. It looks like the main machine talks serial / USB, but not got that playing just yet.

Making PCBs

I designed the PCB on inkscape, which is fine for a small PCB like this. The challenge, as always, is a small single sided PCB with minimal links. My first design assumed I could run tracks between the pins on a 0.1" header (which I could do 30 years ago). It seems this is tricky, to say the least. Maybe we can managed 1/20th" pitch devices like an SO8, just, but tracks between 0.1" headers are not so easy.

So I redesigned with "chunky" tracks. This meant one small link. A 1206 0Ω resistor would be ideal, and Amazon prime do them, but out of stock, so a wire will have to do.

In inscape it is easy to make tracks and pads. You then just stroke-to-path, and union the paths and tracks to make an SVG cut path. Of course this cuts on the track edge which makes the tracks smaller. You can make that a path and stroke-to-path again and union first stage, but that is faff. Simpler to design with chunky tracks and pads that almost touch when designing, knowing the cut path will be thicker. If that makes sense. Obvious I needed a layer for drilling, and cutting around the edge of the PCB, and the tracks.

Making GCODE

This is a lot harder than I expected - there seem to be several solutions, and FlatCAM looked promising, but I cannot get to work on my Mac. There is an svg2gcode tool, but did not work well. I ended up just making my own code to convert EPS to GCODE with options for speed and cut depth and so on. The GCODE goes on the SD card, and simple.

Stupid UI

The UI on the arduino is daft - it has a mode to control portion, but then uses all 8 buttons and has no way to escape back to the main menu as far as I can see - so I set origin and have to power cycle to then run the file. Maybe I am being thick.

The result...

Milling

Drilling

Cutting

Ok I cut too deep and cut in to the bed, idiot. I'll either cut less or add a sacrificial layer below.

That worked

Final PCB

It is not that good, but the design with chunky tracks means it works!

And time to try it out for real...

It only bloody works!!!

P.S. I have open sourced the eps2gcode.

P.P.S. Make sure the screws are tight on the screw ends - else you get drunk tracks on your PCBs.

P.P.P.S. bCNC is the tool for sending GCODE to the CNC machine directly - works a treat on a Mac.

Upgrading the scales to WiFi and RFID

I have now taken things a bit further with my scales upgrade. Not just WiFi, but RFID as well.

This means that instead of pressing send, you tap a card or key fob on the scales.

The scales then send the weight and card ID over HTTPS to a server that puts the details in a database. My previous video showed that I connected a wire to the SEND key for this reason. As it happens it needs a small diode in line so the processor does not get upset at power on, but other than that it works as planned, and was pretty simple.

This is great when the scales are used by everyone in the family, and the next step is some nice graphs and so on. James is working on that.

Of course, whilst this is fun, this suddenly starts to have GDPR implications. For "personal" use we are probably fine, but the second you extend it (e.g. taking to Slimming World to show off, as James is planning) you create fun an games with GDPR. I  can see the "SEND" button being re-labelled "CONSENT" and the privacy policy being labelled on the scales as well.

None the less, a good proof of concept and a fun tech project. I wonder if Slimming World, or Weight Watchers, and Marsden, are interested in teaming up with us and making a complete package. Saves taking laptops to meetings, and the cards are easy to brand :-)

This is how it is done...