Raspberry Pi Node Red Serial Delight

Or – how to get USB serial ports working on the Raspberry Pi

Serial SetupThis originally was a cry for help – I wanted a second serial port on the Raspberry Pi AND I made the mistake of trying out Firmata on the Arduino at the same time – resulting collection of failures had me scratching out my few remaining hairs – but thanks to a lot of work – two lots of changes to the Arduino node itself and some reader insights in here, I can now confidently say that:

1. You CAN add more serial ports to your Raspberry Pi

2. You CAN talk from the Pi (or similar) to Arduino gaining access to ports for things like port on/off, PWM etc. with ease.

It all started when I innocently plugged a USB FTDI into the Raspberry Pi USB connector to give me a second serial port to play with as I was using the first one to talk to a Nextion display.  I checked in Debian in the /dev directory and it up came /dev/ttyUSB0. When I checked in Node-Red it came up with the same – magic. Great start.

I added in the serial node to Node-Red – set up an inject – and a debug node.

I shorted TRX and TX on the serial and… nothing.

(Serial set to a decent baud rate with 100ms timeout).

FTDII could see on the FTDI flashing lights indicating that not only was data coming out  - but also going back in.

I decided to test this and hooked up the internal serial on the Raspberry Pi.  I had mistakenly thought that the graphical control in the new Jessie allowed you to turn the serial login terminal on and off – so I turned off serial. BAD mistake – not only did the serial not work but the GPIO node failed as well!  I turned the serial back on – rebooted, back to square one.

I updated /boot/cmdline.text to remove reference to the serial port… and read THIS -  so now I had serial working.

I tested as you see above.  I shorted TX and RX on the internal serial – and sent some text – smashing – the text shows up in the debug.

So now I connected the serial out from the USB serial – to the serial in on the Pi (ensuring 3v3 output) – MAGIC – that worked.

So all that appeared to be NOT working – was the serial in on the USB FTDI serial – yet there are were error messages.

Is this familiar to anyone? I tried various combinations of the RTS and DTR lines – no difference. All looks fine and connected but the USB0 serial in appeared to do nothing.

Stage 2:

I took out the FTDI - put another one in - that appeared as USB1 (yes, the SAME ONE)... but it worked - I took that out and plugged the ORIGINAL back in - now this appeared as USB2  and worked – but how the hell are you supposed to use a port if it keeps renaming itself. Read on.

I rebooted the Pi (NOT removing power, just rebooted) - it went back to being USB0 - it worked.  I turned the power off, them on - no serial in.

So…. I took the FTDI out - and put it back in - now it appeared as USB1 - adjusted the setting on Node-Red accordingly - it worked.  Rebooted…No USB 1 – looked in the node settings – USB0 was back. Adjusted. All was well on USB0.

I turned the power back off then on (for the whole lot)…. Green connected lights on the in and out nodes, all’s well – BUT – no input. I could again see both lights on the FTDI indicating that it is not only sending but pulling back in – but nothing in the debug window when injecting text.

But this had WORKED after a reboot… but not after power up… so – I tried rebooting again…surely it would work? No.

I pulled the FTDI out – and plugged it back in… USB0 disappeared but USB1 was valid…. I changed the node to work with USB1 – it worked.

So it seems that the only situation when the input would not work – was when both the Pi and the FTDI had been powered down and back up. Which of course is every time you turn the stuff off and on!! And the name was changing – which is just useless.

The Solutions:

Well, thanks to readers in here I managed to split this problem down into sections. The first was this issue of the name keep changing. As it happens this one is RELATIVELY simple.

pi@bedrock2:~ $ sudo lsusb -v | more

Bus 001 Device 005: ID 0403:6001 Future Technology Devices International, Ltd FT
232 USB-Serial (UART) IC
Device Descriptor:
bLength                18
bDescriptorType         1
bcdUSB               2.00
bDeviceClass            0 (Defined at Interface level)
bDeviceSubClass         0
bDeviceProtocol         0
bMaxPacketSize0         8
idVendor           0x0403 Future Technology Devices International, Ltd
idProduct          0x6001 FT232 USB-Serial (UART) IC
bcdDevice            6.00
iManufacturer           1 FTDI
iProduct                2 FT232R USB UART
iSerial                 3 A50285BI
bNumConfigurations      1
Configuration Descriptor:
bLength                 9
bDescriptorType         2
wTotalLength           32
bNumInterfaces          1

So looking at the code above created by running the command at the top above – starting “sudo”…

This told me all about the last USB item plugged in – sure enough my Future technology USB FTDI

I made a note as instructed by reader Alex - of idVentor, idProduct and iSerial and put them into a file:

sudo nano /etc/udev/rules.d/99-usb-serial.rules

The one and only line in that file would be..

SUBSYSTEM=="tty",ATTRS{idVendor}=="0x0403",ATTRS{idProduct}=="0x6001",SYMLINK+="ttyUSB-PETER"

Tried it – failed.  Of course I was trying to use vendor 0x0403 – when I took off the 0x it worked a lot better. I’ve no idea why. With further help from Alex the solution appeared as

SUBSYSTEM=="tty",ATTRS{idVendor}=="0403",ATTRS{idProduct}=="6001",ATTRS{serial}==”A50285BI”,SYMLINK+="ttyUSB-PETER"

All of that was great – on power up the unit would come up with my chosen name – also after reboot – and after disconnecting.

pic

All worked well…one problem apparently solved…

But… from time to time I would lose the output – the lights on the FTDI would show up – but nothing would come back in to Node-Red. Another reader – Christopher had suggested that in some cases you need to short DTR and DCD, also RTS and CTS.  if these were floating and if they WERE important – I could see why I’d be getting strange results. I put links on these.

Voila. Reliable serial connection. It would be nice to clarify why the word “serial” and not “iSerial” – and why you need to miss off the “0x” – but it worked reliably.

Pi and Arduino

Though it works perfectly as a serial port in Node-Red – Node-Red cannot see the new port probably because this is just a shortcut or symbol – if you type it in – it worked – but Node-Red could not (and will not)  search for it – not too much of a hassle to put in manually.

But – the ARDUINO node for Node-Red would NOT accept the manual input.

By specifying ttyUSB0 I managed to flash a light with this Firmata-based node – but any attempts to change things around, try PWM on a pin etc. would result in  Node-Red dying temporarily. I talked to the developer of the Firmata node as I noted the serial node NEVER crashed Node-Red.  Result – thank heavens – a bug – now removed.

Reading a reply in the Node-Red Google group, it was suggested for a while NOT to use the Arduino node but to scrap that and use the contrib-gpio node (the yellow one) – as it happened I had that installed. I used my shortcut (may as well go in at the deep end) – and LO – no more Node-Red crashing.

I tested port digital control as well as PWM and they all worked.. BUT – changing mid-stream – i.e. opening the node and changing the port bit FAILED like the Arduino node – but did not crash Node-Red – this let me experiment a LOT more. I killed the Node-Red service and ran it manually. But at this point, making a change to the node and then running deploy would result in the node sitting with “connecting” and just refusing to connect to the serial port. Simply stopping and starting Node-Red fixed the issue every time.

The author was playing with this and could not replicate this – but it turns out he was using Node-Red’s DEPLOY ALL mode – whereas I just deploy all changed nodes (so my house control doesn’t fall to bits when I’m experimenting). I just had a call to say he can now replicate this.

WHICH MEANS the yellow GPIO node also has this issue! I’ll let them know.

So now that it looks like Firmata will work but does not like change.

Dave at IBM who has been working on the Arduino node has now FIXED this issue in Arduino node version 0.09   (npm install node-red-node-arduino).  I have tried port 23 to make sure the MEGA version of this is running – and sure enough – it is running.

Lovely. Multiple Serial ports on the Pi – and LOTS of GPIO cheap add on – what more could you want. My thanks to Dave for quick action and to everyone for their helpful comments which brought us this far.

The node-red-contrib-gpio node has exactly the same issue that the Arduino node HAD – I’ve left an issue in the repository of the latter.

Update September 32, 2016: I have to say that all of this is working exceedingly well, EXCEPT for Firmata on the Arduino. I left my main Raspberry Pi talking to the USB serial and sending commands to flash a light on an Arduino MEGA2560 board via Firmata. Simple enough... we've had power cuts, I've been experimenting with the Pi etc over all this time since I wrote the blog entry and I can see by the FTDI that once-per-second commands are heading out to the Arduino 24-7 - but every now and then, maybe every few days I look up and that light is not flashing. A quick power cycle on the Arduino board brings it back up - but Firmata is simply NOT that reliable.

Facebooktwittergoogle_pluspinterestlinkedin

35 thoughts on “Raspberry Pi Node Red Serial Delight

  1. Is it really an FTDI or a counterfeit? There have been problems with counterfeit chips that aren't fully compatible with genuine FTDI.

    Try with "cu -l /dev/ttyUSB0 -s 9600". Depending on what flags you to the open call, data reads will block until the handshake pins are valid. That's what the handshake pins are for. A full loopback loops DTR->DCD and RTS->CTS, not just RX/TX.

    1. Hi Christopher - I remember the problems - they were blocked for a while but then that went away... but that's a thought - mine are all the same.

      Right - I'm not entirely sure what those flags you just gave me were - but..

      Here's the strangest thing... with this change, FIRMATA on the Arduino MEGA worked - I could turn output 8 on and off using Firmata.

      but if I selected Analogue output - I would get "connecting" - and then I realised... I was actually connected to the internal serial port - DESPITE the node saying it was connected to the USB version - I could actually turn a light on and off.

      But no more- connect to the actual USB FTDI and I can do nothing.

      BUT - at least Node-Red isn't crashing any more - but by the look of the logs - that's because it can't actually find the board.

      [arduino-board:7bac56fc.e1db38] device cu -l /dev/ttyUSB0 57600 not found. Trying to find board.
      27 Jun 12:21:45 - [red] Uncaught Exception:
      27 Jun 12:21:45 - Error: Port is not open

    2. Ok Christopher, you've solved part of the problem I believe. The cu -l does not work but putting two links on DTR-DCD and RTS-CTS appears to stop the serial from randomly failing - elsewhere I have a solution for the name - blog updated. Thanks for your contribution. I'm sure others will benefit.

      1. Peter, in regard to cu I mean try that from a terminal entirely outside node-red.

        If you can link TX to RX and also link DTR and RTS as above, and use cu (or screen or any other terminal program) you'll be able to prove that
        the hardware itself is working.

        Once you're sure the hardware is OK, we can move on to what is node-red doing wrong. The way to find that out is to use 'strace' to see what system calls node red is making when it opens the port. It may be omitting some flags.

        There's a "CLOCAL" flag that can be set on the serial port to mean "the device is not a modem, ignore the modem status pins (DTR DCD etc)"

        1. With links, FTDI appears to be working perfectly. No matter what I do with Node-Red and the Arduino node - even using the native ttyCOM0 - I can often turn a led on or off but any change, say to another Firmata Pin, Node-Red crashes and restarts. Even if the Firmata software on the ATMEGA2560 was wrong, Node-Red should still connect to the port reliably and not crash. Not heard of strace or how to use it?

          1. Okay, so it sounds like the serial problem is solved, but something else is wrong (more on that below).

            strace is a program to trace system-calls, you run 'strace node-red' and it logs all the operating system access that the application makes. We're looking for calls to "open" and "ioctl" on /dev/ttyUSB0. This could tell us whether node-red is using the special io-control operations to set the terminal flags.

            You can also use stty to view, the port settings while node-red is using the port.

            # stty -a
            # grep Out.of.memory /var/log/kern.log

            I *am* running out of memory from time to time, as I have some first-generation 256M and 512M pis, and the kernel kills the fattest process when this happens, often node-red

            Jul 12 20:51:34 tweety kernel: [209492.223315] Out of memory: Kill process 408 (node-red) score 212 or sacrifice child

            2. is node-red telling you why it crashes?

            # grep Node-RED /var/log/daemon.log | tail -100

  2. Hi Peter,

    Maybe a little bit of topic here. But I found a possible 'bug' or on the parsing of data (still going throught your program to findout why) but if you send via MQTT to the {to_serial2:"AT\"hello"} on the soft serial port only cames out AT anything that is after \" or if you use \x22 (hex of ") is just discarted.

      1. Will do. Just did the test using the real serial port by sending the same as what i'm sending over MQTT and it does send the correct information. So if i send {to_serial2:"AT\"hello"} via MQTT i get = AT
        {to_serial2:"AT\"hello"} via serial port = AT"hello

  3. Greetings.

    Without a careful review of all the bits and the changes you made, my first guess overlaps with Linux permissions and ownership of the /dev/ttyUSB0 device.

    Perhaps by default your user did not have rights to the device, but after some change to one of the settings you had the needed rights -- and those were lost again after the "cold" restart.

    I don't have a Pi on the desk at the moment to help add to the debug, but some combo of "ls" commands may clarify, or doing some of your tests as root (via su, or sudo, etc.) may help spot if it is a simple issue at the device level.

    --Andrew

    1. I would have THOUGHT (possibly incorrectly) that there would be no difference between a restart and turning off and on. Root is a bit of a problem, having setup Node-Red as repeatedly advised as PI and not root I've always tried my best to do everything as Pi. Not sure if you can simply go into /dev/ttyUSB0 and change permissions. It does seem odd to me that if you unplug the device and plug back in - it ends up as USB1 device - seems daft behaviour for the same device. In Windows, a given device would always appear as the same COM port. I'm awaiting delivery of some slightly different FTDIs just to make sure it isn't the device itself. I'm convinced there are two problems - firstly something to do with the FTDI - and then to do with the Arduino node as it is definitely not disconnecting when you make a change to the settings.

      1. The unplug thing is usually down to a process holding the device in an "open" state, even though the physical device itself has been removed. When the kernel sees the original device plugged back in again it assigns it a new device node, as the original (USB0 in your case) is still in use.
        I suspect that the power-on issue is a chicken and egg situation between your FTDI adapter's looped-back pins, given that it takes the kernel a while to getting around to initializing the physical device during a power-cycle boot (whereas it has already been previously initialized in a plain, non-power-cycle reboot).
        Despite the flashing lights, it's probably sat there in some indeterminate state, not knowing whether it should be transmitting or receiving (summatlykethat, anywaze 🙂 ).

        1. PuceBaboon said "The unplug thing is usually down to a process holding the device in an "open" state, even though the physical device itself has been removed. When the kernel sees the original device plugged back in again it assigns it a new device node, as the original (USB0 in your case) is still in use."

          My sense as well. It's probably node-red that had a hold of the original port and caused the increasing port #s.

          I did a little googling and things get more interesting with the Pi 3 when trying to use the on-board UART due to the addition of the bluetooth. Be sure to research that before trying to do serial with the Pi 3. Hopefully you can resolve this issue and you won't have to mess with that.

      2. chmod a+rw /dev/ttyUSB0 will allow read/write permission to everyone. or add the pi user to the 'dialout' (sudo adduser pi dialout) group - that seems to be the default group for the ttyUSB's on my relatively recent Raspbian PI2.

        dmesg (maybe sudo dmesg | more) will show you all the boot-up information. You should see it initializing the serial ports. Something might point you in the right direction there if it's not as simple as permissions.

        You can give persistent(and custom) names to serial devices under linux. There's a few steps involved, but it's not rocket science. Here's a link - I haven't tried this one personally, but if it doesn't work, there should be enough info to google for a different procedure. It's kind of nice to point to /dev/arduino when you are talking to your arduino 🙂

        https://www.element14.com/community/community/design-challenges/sci-fi-your-pi/blog/2015/07/15/persistent-names-for-usb-devices

        Now back to playing with my new oDroid XU-4 Octocore 2GHz ARM board with 2GB RAM.

        Jerry

        1. hi there Jerry - that Odroid XU-4 looks like a powerful beast and way better than its more expensive and now defunct predecessor. What do you think so far of it?

          On the Raspberry Pi, user PI is already a member of the dialout user group.

          pi : pi adm dialout cdrom sudo audio video plugdev games users input netdev spi i2c gpio

          In fact when I use other boards I make a user called PI with the same permissions.

          Interestng - you've taught me a new command - I'd never used DMESG before and it does indeed produce some interesting reading - I'm just not sure what to do about it - the USB FTDI is plugged in - and..

          [ 1662.331691] ftdi_sio ttyUSB0: usb_serial_generic_read_bulk_callback - urb sto pped: -32
          [ 1663.388083] ftdi_sio ttyUSB0: usb_serial_generic_read_bulk_callback - urb sto pped: -32
          [ 2274.741205] gpiomem-bcm2835 3f200000.gpiomem: gpiomem device opened.
          [ 2274.825206] gpiomem-bcm2835 3f200000.gpiomem: gpiomem device opened.
          [ 2274.885015] gpiomem-bcm2835 3f200000.gpiomem: gpiomem device opened.
          [ 2274.902925] gpiomem-bcm2835 3f200000.gpiomem: gpiomem device opened.
          [ 2275.609186] gpiomem-bcm2835 3f200000.gpiomem: gpiomem device opened.
          [ 2292.693240] ftdi_sio ttyUSB0: usb_serial_generic_read_bulk_callback - urb sto pped: -32
          [ 2299.600777] ftdi_sio ttyUSB0: usb_serial_generic_read_bulk_callback - urb sto pped: -32
          [ 2429.476096] ftdi_sio ttyUSB0: usb_serial_generic_read_bulk_callback - urb sto pped: -32
          [ 2432.888688] ftdi_sio ttyUSB0: usb_serial_generic_read_bulk_callback - urb sto pped: -32
          [ 2579.013597] gpiomem-bcm2835 3f200000.gpiomem: gpiomem device opened.
          [ 2579.108278] gpiomem-bcm2835 3f200000.gpiomem: gpiomem device opened.
          [ 2579.137431] gpiomem-bcm2835 3f200000.gpiomem: gpiomem device opened.
          [ 2579.180113] gpiomem-bcm2835 3f200000.gpiomem: gpiomem device opened.
          [ 2579.908765] gpiomem-bcm2835 3f200000.gpiomem: gpiomem device opened.
          [ 2626.334361] ftdi_sio ttyUSB0: usb_serial_generic_read_bulk_callback - urb sto pped: -32
          [ 2629.976833] ftdi_sio ttyUSB0: usb_serial_generic_read_bulk_callback - urb sto pped: -32
          [ 3016.805360] usb 1-1.4: USB disconnect, device number 6
          [ 3016.805789] ftdi_sio ttyUSB0: error from flowcontrol urb
          [ 3016.806281] ftdi_sio ttyUSB0: FTDI USB Serial Device converter now disconnect ed from ttyUSB0
          [ 3016.806364] ftdi_sio 1-1.4:1.0: device disconnected
          [ 3018.837627] usb 1-1.4: new full-speed USB device number 7 using dwc_otg
          [ 3018.965192] usb 1-1.4: New USB device found, idVendor=0403, idProduct=6001
          [ 3018.965213] usb 1-1.4: New USB device strings: Mfr=1, Product=2, SerialNumber =3
          [ 3018.965224] usb 1-1.4: Product: FT232R USB UART
          [ 3018.965235] usb 1-1.4: Manufacturer: FTDI
          [ 3018.965245] usb 1-1.4: SerialNumber: A50285BI
          [ 3018.973562] ftdi_sio 1-1.4:1.0: FTDI USB Serial Device converter detected
          [ 3018.973775] usb 1-1.4: Detected FT232RL
          [ 3018.974672] usb 1-1.4: FTDI USB Serial Device converter now attached to ttyUS B1
          [ 3049.181856] ftdi_sio ttyUSB1: usb_serial_generic_read_bulk_callback - urb sto pped: -32
          [ 3052.854712] ftdi_sio ttyUSB1: usb_serial_generic_read_bulk_callback - urb sto pped: -32
          pi@bedrock2:~ $

          1. This post describes the same error message, and a simple fix, as long as you don't need fast USB access. The Pi ethernet is attached to the USB - suspect this fix might make the ethernet run at USB 1.1 speed as well. It'd be worth a try to test.

            https://www.raspberrypi.org/forums/viewtopic.php?p=284594

            Given the symptoms of it only failing after a complete power off, you might try resetting the USB bus to see if it helps...then in your rc.local you can reset the bus after power on to workaround the problem and still have usb 2.0 speeds.

            This post discusses that.

            https://www.raspberrypi.org/forums/viewtopic.php?p=284594

            Note the last post indicates the path to write to has changed - on my recent Pi's it is now: /sys/devices/platform/soc/3f980000.usb/buspower

            As with most other things in linux, this info is presented as a file, so you can just 'cd' and 'ls' your way through the tree to confirm the file name in your system. Again, because the ethernet and USB share the same bus, when you drop USB bus power you will lose ethernet which would be unfortunate if you were remotely connected to the Pi vi ssh. Hence the script, which will run without USB or Ethernet connected.

            And of course, if you have another dongle that works,, you could use that one and print a label that says 'don't use on pi' and attach it the the non-working dongle and throw it back in the drawer 🙂

      3. Currently we are trying to run twould arduino boards off of the pi. One arduino board connects flawlessly using the symbolic link. The other I have to switch between the /ttyACM port it is on and the symbolic udev link I've made for it on just about every run. I have tried both the arduino node and the gpio node. Is there a way to get the arduino node to always look for the symbolic link and connect every time?

  4. Perhaps, you need to have a persistent USB (USB0)?

    Run the following command:

    sudo lsusb -v | more

    and write down the:

    *idVendor
    *idProduct
    *iSerial

    Create a rules file, with the following example content:

    sudo nano /etc/udev/rules.d/99-usb-serial.rules

    For example:
    SUBSYSTEM=="tty", ATTRS{idVendor}=="0658", ATTRS{idProduct}=="0200", SYMLINK+="ttyUSB-ZStick-5G"

    Reboot and enjoy

    1. Alex - nearly there.. and thank you for helping me get this far....

      This is what I actually from your command...

      idVendor 0x0403 Future Technology Devices International, Ltd
      idProduct 0x6001 FT232 USB-Serial (UART) IC
      bcdDevice 6.00
      iManufacturer 1 FTDI
      iProduct 2 FT232R USB UART
      iSerial 3 A50285BI
      bNumConfigurations 1

      and this works..

      SUBSYSTEM=="tty",ATTRS{idVendor}=="0403",ATTRS{idProduct}=="6001",SYMLINK+="ttyUSBPETE1"

      I have a couple of questions I hope you can clear up.... so - why did I have to remove the 0x for these two to work???

      Secondly - I tried the serial number - firstly based on the above I tried the utterly useless "3" and that failed - then I tried "A50285BI" and that failed too.

      Clearly this works but I assume now that 2 models from the same manufacturer could not co-exist - is there a way to incorporate the serial number?

      1. Peter,

        Apologies It was a quick reply - you have to remove "0x" in idVentor and IdProduct when you insert them into 99-usb-serial.rules. I forgot to mention it. Some products do not have a serial number - if you do not have it or struggling to find it just ignore it. However, every USB device has a Vendor ID and a Product ID as seen in lsusb output - these must be inserted. If you have several similar FTDI chips like myself, the Vendor ID and the Product ID may match. In this case, you must use the serial number. For FTDI (and perhaps for other USB chip vendors) it is 8 digits (for example A7004IXj).

        Unfortunately, the USB device names are not persistent in Linux. That's the way it is.

  5. For you the 99-usb-serial.rules should be

    SUBSYSTEM=="tty", ATTRS{idVendor}=="0403", ATTRS{idProduct}=="6001", ATTRS{serial}=="A50285BI", SYMLINK+="ttyUSB-PETER"

    Do not forget to reboot or issue the following:

    sudo udevadm control --reload
    Then disconnect and reconnect the USB device, to see if it works.

    1. By the time I was finished understanding the flags etc I was pretty happy with the USB serial on the Pi - mind you - if you need serial ports - some of the Pi alternatives offer a lot more than one serial port.

  6. Hi, I am using node-red in Rasbian Jessie. I am able to receive data from serial port (using debug and serial in nodes) but I am not able to send data out on serial port.( I used inject and serial out nodes for the same)

    1. Now - that's interesting as last time I looked they'd completely screwed up the settings on Raspbian (Pi2) for the serial port - when I turned it OFF - I thought I was turning off serial access to debug - turns out I turned off all GPIO and serial access. Might want to have a play with that.

Leave a Reply

Your email address will not be published. Required fields are marked *