Notice I titled that remote access as against remote programming – but that’s part of it.
Years ago, having gotten sick and tired of dismantling my home control Arduinos just to upgrade the code, I purchased a couple of small units from China which I’ve just thrown in the bin. One had a USB plug on it and plugged into the PC, the other had an FTDI output on it. Both had those black 2.4Ghz antenna.
So the purpose of these units was to wirelessly replace an FTDI – or serial programmer, the idea being that instead of bringing the Arduino project back to wherever one programs the devices, you plugged the unit into the project and over the air programmed it.
At the time I was using Arduinos and those CHEAP £1.50 radio modules – the NRF24L01s, the worst radio boards in the world who’s range could be so bad you’d be lucky to get through one wall. I had high hopes for them with a great mesh network called RADIOHEAD, but ultimately their range and lack of ability to read signal strength doomed them. However, as you can imagine they were dotted all over the house and I thought this little radio pair would solve the issue of programming them without ripping everything up.
It turns out the “radio FTDI” had the same problem – it was based on the NRF24L01.
Fast forward to September 2015. Here I am in Spain with a new setup of ESP-12 modules controlling lights, watering system and a lot more. I’ve made some improvements to our “home control 2015” software in the ESPs and once again I find myself having to dismantle kit.
Now, I know that is it possible to program over the air – a few guys have tackled this successfully and indeed much of the code is built into the Espressif SDK. Right now my friend Aidan is working on adding this to our code but there are two issues, he’s not had luck up to now and secondly as our code comes to something like 300k you’re never going to be able to add this to the ESP-01 modules (we use ESP-01 sometimes, ESP-12 other times…) as you need as much spare storage as you have code for OTA programming!
It was with that in mind that last night I dragged out the old FTDI boards to see if I could save myself the effort of dismantling the Pergola lighting system to update it. Suffice it to say that I managed to get a few bytes programmed before that all came to an end despite several attempts. With around 25 WIFI units of one description or another dotted around the place, the old wireless FDTI (same frequency range) simply could not handle more than a few feet.
Bin.
I figured it would not hurt to look around and see if someone has alternative wireless solutions – lots of time has passed since I bought that kit.
And so it was that I stumbled upon this. https://github.com/jeelabs/esp-link – as it happens it is quite new – essentially what we are looking at is a single unit (yes, that had me going for a while) which will hook into the WIFI system and act like an FTDI – given that you can fire the serial out of your PC over TCP to a port – port 23. I grabbed the code (the stable 1.04 version) and very excitedly blew it into an ESP-01 unit I had lying around.
Really, nicely done web interface (with remnants of Martin Hanzarov’s code in there) and a really good write-up – indeed there may be enough info in there to help us implement our own FOTO (over the air) updates to our code.
But this has far wider implications – a virtual serial link would also let you monitor the serial line of a remote unit for debugging info – and our units do indeed put out lots of serial debugging info. It would also let you remote program and monitor un-wirelessly enabled Arduino projects.
So the first problem was – I had this unit sitting as 192.168.1.27 on my network and I needed to blast info into port 23 to have it come out as 115K serial. I have ECLIPSE sitting on the PC running my projects and there’s a line in the MAKE file that says “COM6”. I have absolutely NO idea how to turn that into a TCP statement if indeed it can be done (FEEL FREE TO START TYPING IF YOU KNOW HOW).
So I turned to a 30 day trial of a PC program that promised to provide a “virtual com port” – and that it did. Serial/IP Redirector let me create a link to 192.168.1.27 port 23 and ship raw data back and forth to COM22 (which doesn’t exist on my PC)- LO AND BEHOLD I set it running, put PUTTY on the PC talking to COM22 and with a real FTDI attached to the little ESP-01 board on real COM6 I set that running on a terminal –at 115K. Utterly flawless raw serial communications – rock solid.
I was getting excited by now. I took my ESP-01 board and cross-connected the serial to an ESP-12 test board. I held GPIO0 to ground and turned the power on, putting the ESP12 into programming mode.
I punched COM22 into the ECLIPSE project and hit the FLASH button on the project. Lights flashed on both units. With a scope attached to the serial in on the ESP-12 I could see clean 115k data going into my project and… nowt, zilch, nothing.
I was up until the early hours ensuring the baud rate was ok and that my level conversion wasn’t getting in the way – you see all my projects use a resistive divider on the serial input of ESP boards so the whole lot will work just fine with 5v logic on the FTDI and elsewhere. I wondered if hooking two of these units together with serial cross connected might prove too much but the scope says no – it says I’ve a perfectly reasonable serial signal going into the ESP-12 – but it’s not having it – no responding output whatsoever. Eclipse tries sending out data over and over and eventually gives up just as if there’s nothing connected.
So near and yet so far.
I got up this morning and the unit was still rock-solidly connected to the WIFI and typing a few characters provided perfect results at the other end – but I cannot program ESP units – yet.
Very exciting, yet very frustrating.
Perhaps coffee and a new start to the day will help. BUT I had to bring this to you as the article is really good, the software seems to WORK and there’s a super-bumper version with it’s own OTA updating in the works for larger targets like an ESP-12 (more FLASH) than the ESP-01.
I’m sure this is going to be a winner – just need to get it to work. After all, the ability to add remote programming to any FTDI-enabled project for what, a couple of quid.. well, you just can’t ignore it.
Meanwhile if anyone reading this firstly knows how to set COMX on Eclipse (on Windows) to TCP – do let me know – and if you really think you know what I’m doing wrong here and want to jump in before I figure it out – by all means – you have the keyboard…
I don’t think I tried the version 2 beta. I did have lots of issues getting a stable (free) Com port connection on the PC.. but anyway, I really do like the idea of 2 ESPs then you’re not reliant on any PC as such – just serial in one end, serial out the other – preferably with a couple of handshake lines to boot.
Since one of your issues was OTA programming, I do notice that version 1.0.4 is a “Release with over-the-air (OTA) reflashing of the esp, port 23 transparent bridge as well as AVR & ARM flashing support.” https://github.com/jeelabs/esp-link/releases/tag/v1.0.4
If you were running an earlier version, you may want to have another go at it with that release (September 15)
I haven’t been able to get any of the Release 2’s to load onto the ESP-12 that’s on the Adafruit Huzzah. Not sure why. When I receive the ESP-13 this weekend, I’ll try loading Release 2 RC1 onto the ESP-13, and I expect it will work.
Yup I’ll give it another go – some other issues to address first – went off to the coast for 2 days – the trip went on our power and within hours my trusty uninterruptable power supply ran out of joice. I am SO annoyed this item is old – maybe 2 years or I’d be ruining their Facebook page – would you believe when the power came back on – the “uninterruptable” power supply who’s SOLE job is to run un-attended – DIDN’T come back up and had to have a button pressed to start it up!!!!
I received my first ESP-13 and was able to load esp-link Release 2 RC1 onto it without issue. Adding a pull-up resistor to the ESP-13’s RX pin definitely helps. Perhaps that alone will solve your issue!
Were you ever able to resolve this?
NO I put it on hold pending WIFI issues… I’ve noticed – but not even looked at in depth – ESP-ARDUINO environment – has a TCP to serial example – so a telnet client I guess can send values – and they’ll come out of the serial pin. There is also (sadly not specifically ESP) a CLIENT example. Now it occurs to me that if one ESP was running the server and the other the client – between the two of them you should have transparent high speed serial – put your USB-serial converter on one of them – and the serial should flow from whatever programmer out of the other ESP – only problem there – when programming Arduino and SOME esps – you’re also using a hardwire to reset the thing – you can’t trigger that with serial as you could be sending any of the 256 characters… I’ve not looked at this yet but if anyone wants to give this a go with a couple of ESP12s….. it would be great if we could do this reliably – in my case instead of having to get ladders out or get the OTA running (which I can’t get reliable) I could simply plug one end into my project – go back into the lab and program it from the PC!!!
Which version of esp-link are you running? It appears that Version 2 RC1 running on an ESP-13 (4MegaByte Flash) will be the ideal setup.
Maybe this will work for you?
@Pete
If you’re still stuck, I suggest you do a loopback to confirm the ESP12 is at least receiving exactly the same bits you think you’re sending it via your virtual serial connection. If it is, then maybe try sending just a tiny sketch to see if that works, so as to eliminate the possibility you’re running out of memory for some reason.
A suggestion: COM0COM (http://com0com.sourceforge.net/)
This is a gem I found long time ago and very powerful. It lets you create virtual com ports and through command line you can manage and redirect the traffic to other virtual com port o tcp port. Very useful for debugging serial connections. You can even create tunnels between serial ports over tcp.
To get it working on Win8 I had to dig a little to find signed version of drivers or disable signed driver req in windows. I think I found signed version at github but not sure…
I built an ESP8266 / arduino hex code uploader that worked via port forwarding.
It was a website where you could drag a hex file, type an IP, and it would upload the code.
The backbone of this was this line of code :
“avrdude -c avrisp -p m328p -P net:’.$ip.’:23 -F -U flash:w:’.$FILE.’.hex”
How about this for COM port IP
https://github.com/smslib/comm2ip
Binaries available from :-
http://smslib.org/download/ (bottom of page)
Not used this, just something I found.
What a hero. I just got back and noticed this and tried to give it a shot – at first I got all manner of errors and was just about to give it the thumbs down when I realised it was likely to need administrative rights – and so popped up the DOS box in Windows 10 as admin – and after setting the port etc in the config file tried to run the service. Worked a treat first time – I currently have the serial info coming in on COM22. Now, I’d done that before but with a commercial program – this is a freeby. Thanks Andy – well spotted.
Argh, spoke too soon, it was the commercial product sitting in the background working (and that is too slow to use as a programmer). Comm2IP bombs if you use anything other than 127.0.0.1 (at least on Windows 10) – I’ve left an issue with the developer.
Shame. Looking at the source code Comm2IP seems to provide a local listening TCP/IP endpoint which is mapped to a local COM port. So it can’t connect to remote devices over TCP/IP, the remote devices have to connect to it.
I’ve been messing with something similar but using SoftSerial on an Arduino mini pro (pins 10 and 11 in my case) but I’m not attempting any re-flashing.
The 5v Tx from the Arduino is sent through a voltage divider to the Rx of the ESP-01 (good)
I have the 3.3v Tx from the ESP-01 connected directly to the Rx pin on the Arduino (not so good)
I can only get this to work at 9600. … some thoughts….
Does the ESP properly pull the Arduino into flash mode?
Does the ESP need a power reset after flashing?
Could you use a 3.3v mini pro?
You could try 4-channel Bi-Directional Logic Level Shifter for all signal connections http://www.ebay.co.uk/itm/4-channel-Bi-Directional-Logic-Level-Shifter-Converter-3-3V-5V-For-Arduino-/321763314096?hash=item4aea9689b0
Not relevant here and now but I read that the Arduino time shifts between Serial and SoftSerial… I wonder if the ESP effectively does the same?
Isn’t it just worth replacing all the ESP-01 with ESP-12 (reusing then on another project) and using Richard Burton’s fantastic RBOOT code rather than spending lots of time on this? 🙂 I swapped out the ESP-3s that I was using previously.
I now have RBOOT working really well on my IoTBox project. Richard was really helpful too.
Otherwise: Could it be a buffer problem – there is not much ram on an ESP?
Erm, you’ve lost me there – I DO use Richard’s RBOOT – have done for many months?
But the buffer comment could well be valid – not that much RAM
I was wondering if you could use the ota update built into rboot if you were only doing code updates?. I use it for updating 35 IoTboxes around Europe.
WHAT?? I didn’t know RBOOT had an OTA update built into it???? Erm, well, tha’ts great and do tell me more but of course it would still not do it for any board that didn’t have more than double the memory I am using – but by all means please DO go on – I had no idea there was that facility built into rboot?
Yes, rboot-ota. Check out my blog to see how I use it and email me your details and I will add you to my private bitbucket. My code is a bit rough hence not public yet. Esp12 set to 2 x (webpages and code) + 1 x settings storage. Pulls updates from external webserver.
Oh, you still have the arduinos. Sorry, thought that they had been replaced.
Noooo I’m not using Arduinos? I’m generally using ESP-12 boards – using Richard’s RBOOT. You have my undivided attention. Imy cide extends from the bottom (a little bit for rboot – then my code all in one section, then a little section near the top for web pages.)… So I’m using up to 4091c – ie just over half way through the 512k.,,, and load points are 0, 2000 and web pages at 70000 (all hex)…. I’m getting excited now – the code is all in C incidentally…
I am doing this here: http://www.instructables.com/id/ESP8266-WiFi-File-Management
Although I’m using NodeMCU and PHP, the process is simple and can be adapted to other firmware. This can also be done via MQTT. MQTT offers some great advantages but also the extra considerations of broker management.
Yes, easy remote management is the key to making the ESP practical.
Cheers