Author Archives: jwothke
Starting with a little retrospection: So far I had built two sprinkler controllers that I remotely control via a home grown (Java) PC software (see my earlier sprinkler posts).
These devices are based on a cheap ATmega128 micro controller which requires a separate breakout board and a bit of soldering to get going. With some custom PCB this would actually be a neat chip to use – with more IO pins than you’ll probably ever use. But with the breakout board it uses quite a lot of space and it gets very annoying when one of them burns out (e.g. due to some surge on the power grid) and you quickly need to come up with a replacement.
The devices then use a Si4432 transceiver to communicate with the PC application. The Si4432 theoretically should have an impressive “best case” range and is fairly inexpensive too. As always, solid obstacles (like walls) are the limiting factor for the real range of this transceiver: In my case the obstacles are two tiled roofs and I am ending up with a maximum range of around 50m-70m.
However I soon found, that the Si4432 is pretty fragile (i.e. these things just keep dying for no obvious reason) and most of the really cheap PCBs use 1.27mm pin spacing which makes it a pain in the ass to replace them or to even hook them up to your 2.54mm prototyping boards. I had therefore opted for a somewhat more expensive version with 2.54mm pin spacing. Unfortunately there seem to be different builds (I already saw three) of very similar looking (blue) PCBs, and to make things worse some of these are just incompatible. So when I wanted to just add one more device to my existing zoo I very annoyingly ended up replacing all existing transceiver PCBs because the new version that was sold at that time no longer worked with the older models I already had. Also transmission speed of the Si4432 is not that great (to some degree that may be my fault since I traded in some speed in favour of a more reliable connection).
Based on the above experiences I felt that I should try something new for the 3rd device that I was about to build (something to be used in my greenhouse). I decided to give WiFi a chance after all and replace the ATmega128 & Si4432 with a “Wemos D1 ProMini” (ESP8266 based). The idea is that it should be much easier to get some standard WiFi range extender if necessary and the transmission speed should be much higher in any case. The respecive board costs around five bucks (with external antenna & cable) so it won’t hurt if ever there is the need to replace it.
Obviously I will keep the old devices that I already have and I therefore decided to extend my existing software such that it also is capable to deal with WiFi/UDP based communication. I just completed the respective software changes and everything seems to work like a charm (and blazingly fast):
I still have drill some holes in the device case (to install the external connectors) and arrange for some 12V DC power in the greenhouse – then it is ready to go 🙂
For now just these ESP8266/Wemos D1 related observations:
- In principle, migrating existing Arduino code to the ESP8266 is straight forward: Most of the existing libraries that I had been using previously, also work on the ESP8266 (I did only have an issue with one specific EEPROM library).
- There is one significant difference to a “normal” mircoprocessor: WiFi handling (i.e. the built-in behind the scenes functionality) is the ESP8266’s first priority.. Your application code IS NOT and it MUST NOT get into the way or the device will just crash (there mustn’t be any long running sections in your code.. “yield()” generously – this is the annoying part: having to litter your code with ESP8266 specific yield() code that has absolutely nothing to do with your application logic ). A long running “setup()” may also be a problem and you may have to move setup code into the “loop()”.
- Debugging an ESP8266 is no fun (make sure you always have the means to inspect the exception output that the ESP8266 libs dump to Serial!). The reference here may be useful. When crashes are triggered by the ESP8266’s timing constraits, the behaviour can get quite unpredictable and it may be difficult to locate the root cause (I did not use any debugger tools with the Sloeber IDE that I am using).
- Flashing a new software onto the device has its pitfalls: It seems that for some reason the “reset” does NOT ALWAYS properly reset the device: A device that does not seem to work after reflashing suddenly works after a power-off. (I also had issues with the attached I2C/LCD module: Sporadically it would crash so badly that the device had to be powered off completely for 30secs to bring it back to life. There were some occasions where I thought that something was wrong with my updated code when in fact it was just the LCD that was no longer updating..)
- The Wemos D1’s micro-USB connector is junk.. aka one big lose contact.
- Wemos D1 IO pins are scarce – use I2C whereever you can to preserve the little that is available. (This may become a pressing issue in case you also intend to use SPI – which may be a prerequisite to hook up things like an SD-card reader.)
I recently noticed that ’emscripten’ meanwhile allows to also generate WebAssembly output. WebAssembly is touted (see e.g. https://hacks.mozilla.org/2017/03/why-webassembly-is-faster-than-asm-js/) to use less space and to run more efficiently (i.e. faster) than previously existing web technologies.. sounds great!
With my portfolio of various ’emscripten’ compiled chiptune music players this seemed like the perfect opportunity to just give it a try (If you want to try this yourself, make sure to really get the latest ’emscripten’ version! Also be warned that the new ‘clang’ version that ’emscripten’ is using now, is more strict with regards to existing C/C++ standards and you may need to fix some of your old bugs in the process..).
Due to the fact that web browsers will load the *.wasm WebAssembly files asynchonously, existing bootstrapping logic may need to be reworked (you’ll need to wait for the notification of the loaded ’emscripten’ module that it is actually ready – don’t even think about using the SINGLE_FILE hack, it won’t work in Chrome!).
In the case of my chiptune players, migration fortunately wasn’t a big deal (my player was already prepared to deal with asynchronously loaded stuff) and soon I had the first *.wasm results. And from a size perspective, those output files already were good news: In their old asm.js incarnations some of my emulators are rather bulky and in total the size of the nine emulators originally summed up to more than 11MB. The better optimizer used in the new ’emscripten’ already managed to bring those asm.js versions down to about 10MB – but with *.wasm that now shrinks to 5MB. Nice!
I then went about measuring the performance of the different versions (I tested using Chrome 64 and FireFox 57 on a somewhat older 64-bit Win10 machine). I was using my all-in-one “Chiptune Blaster” page as a testbed (see https://www.wothke.ch/blaster/ and https://www.wothke.ch/blasterWASM/). I patched the music player “driver” to measure the time actually spent within the various emulators while they are generating music sample output. I started measuring after each emulator had already returned some sample data (i.e. its program code had already been used) and then measured the CPU-time that it took to generate 10 seconds worth of sample data (i.e. the numbers in the below table are “CPU ms / sec of music output data”, i.e. smaller is better):
I repeated my measurements multiple times (6x) and eventhough the results were – for the most part – reproducable, they fluctuated considerably (e.g. +/-10%). Any single digit percentage measurement is therefore to be taken with a pinch of salt. In Chrome there were even some massive hiccups (maybe some background garbage collection? see “(*) worst times” in parenthesis). The above table shows the “best” result that I ever observed for the respective scenarios.
Interestingly with regards to the “better performance” claim, the results are not really conclusive yet. There are some finding though:
- Chrome users may typically experience a massive performance improvement from WASM.
- FireFox’s asm.js implementation already performs much better that Chrome’s. For Chrome users, WASM here is actually only the 2nd best choice – for most scenarios the performance benefit of switching to FireFox here is even bigger.
- For FireFox users the situation here is more complicated. It really depends on the specific program: Some may run massively faster, but some may actually run slower than their asm.js equivalent!
PS: I had only briefly looked at Edge but asm.js performance is slightly worse than Chrome’s and WASM is almost 2x slower than Chrome’s.
An important thing that I did not mention yet are startup times: WebAssembly is designed to be parsed more easily than respective JS code, the asynchronous loading may then also speed things up (in case your browser really puts those multiple CPUs to good use..).
And indeed this is where Chrome (and even Edge) actually shines: For the old asm.js version of my page it takes about 3 seconds for Chrome (4 seconds for Edge) to locally load/display it on my PC. For the new WASM version it’s barely more than 1 second (also for Edge)! FireFox somewhat disappoints here: It also improves on the 4 seconds for the old asm.js page, but the new WASM version still takes 2 seconds to load/display (Chrome/WASM may not be too bad after all.).
- So WebAssembly may not always improve execution speed, but combined with the greatly improved startup time it is really nice!
This is just a little update regarding my earlier DIY sprinkler project: The old Rainbird product (see left photo below) had finally died completely and the time had finally come to put in “version 1.0” of my home-grown replacement (see photo on the right).
As compared to the original post I had meanwhile replaced the ATmega328P (i.e. the Arduino ProMini) with an ATmega128: This microprocessor gives me more space for the program code – but it still is a very cheap IC.. However it comes at the price of some extra soldering (see green PCB on the photo above). Anyone interested in using this chip as a replacement for less powerful Arduino’s should have a look here: https://github.com/MCUdude/MegaCore
Previously I had been performing all of my testing with brand new 12V DC solenoid valves. The question was whether or not my new 12V DC controller would also work for the old 24V AC valves already in place from the old installation:
To my great relief it works perfectly and I can finally control the things remotely from my PC 🙂
In addition to showing some cool raymarching based realtime graphics, my latest web page is dedicated to “269 Life” and the matching title is meant to attract some extra attention to that meaningful movement (see http://www.269life.com).
The realtime WEBGL web page can be found here: https://www.wothke.ch/269life/. You’ll need some sort of 3D graphics accellerator and a Chrome browser to use it. Or you can have a look at a youtube recording here: https://www.youtube.com/watch?v=Y3S3MY3Vuf4 (I am still looking for a volunteer to create a higher quality recording for me 🙂 )
I am not repeating the information that can already be found in the comment of the youtube video here. Instead I’ve added some background information regarding the techniques that I used in the page.
All the fractal graphics are created using knightly’s “pseudo kleinian” algorithm (see example code in “Fragmentarium”) as a base and parametering it with various “distance estimate” functions. An “orbit trap” based implementation is used to color the result. Depending on the specific “scene” a number of reflections is calculated (up to three). The “phong blinn” shading model is finally used in combination with a standard “ambient occlusion” implementation to render the fractal background (basically the same impls that I had previously used in “modum panem”).
Three different approaches are used to display text elements:
- “Flat” texts are created by using a font texture that is then displayed via simple triangles (two per character).
- Texts like the title or the ones in the “greetings” section are then based on extruded 3d fonts (see standard THREE.js examples).
- Finally there are the “particle based” texts that explode in the “greetings” section – which are created using regular canvas text rendering.
A “bokeh” postprocessing is applied to the resulting page to create a “depth of field” effect. (The respective implementation is derived from Dave Hoskins work.) The “bokeh” postprocessing is also used to create some interesting distortion effects on the overlayed title text (which is not using the same z-buffer).
Finally the “greetings” scene showcases the combination of “standard” THREE.js elements (particles, extruded texts, etc) with the shader generated fractal background: By having the fractal shader propagate its z-buffer information, the regular THREE.js overlays are later clipped correctly (thanks to Marius for the respective depth calculation – see boxplorer2).
The “neon signs” here are created via a postprocessing pass that adds a “glow” as well as a “god’s ray” effect. A simple random noise based shader is used to create the purple “northlights” on the horizon and 20’000 confetti particles provide for some action.
Thanks again to Wolf Budgenhagen and Darkmelo for letting me use their music.
Obviously there was something missing in my previous mandelbox experiment… exactly, some means of transportation so you can explore the scenery!
I therefore extended the original version such that “regular” 3D stuff can be mixed with the raymarching based fractal landscape.
also I added a bit of “collision detection” so that the landscape can be moved “out of the way” when your transport gets too close..
so enjoy your ride: https://www.wothke.ch/modum/#/wright-and-bastard/sets/evoke-2016
or have a look at the youtube recording: https://www.youtube.com/watch?v=bx0Aakv3JPE
Notice: Unfortunately Chrome seems to be the only browser that currently properly supports WEBGL_draw_buffers. (Firefox also claims to support the feature but in fact it is completely messing up the respective color attachments..
And to complete the trilogy here another bit of fractal fun (in case your PC is not equipped with a fast enough graphics accelerator you can find a video recording here: https://youtu.be/PzTwOfoZp0E):
I just did a little revival of my old WebGL fractal stuff. The two below screens use the latest version of my Commodore C64 emulator to play two fitting music creations by Markus Klein.
I have some older automated lawn sprinkler installed in my garden. So when the respective controller started to misbehave my first reflex was to try and repair it.. but not having any real clue about electronics, this proved to be somewhat difficult, and with the limited functionality that the commercial product had offered in the first place, I reckoned that it wasn’t worth the effort anyway.
But being tired of performing the same manual irrigation round every one or two days during summer – not to mention the vegetable garden – I thought that it might be a fun little project to design a replacement tailored to my needs, while learning a thing or two about simple electronics.
What I wanted was a system that would not just stupidly irrigate based on some predetermined schedule, but a system flexible enough to adapt based on actual irrigation needs. Also I wanted to be able to collect (on my PC) all weather/irrigation related data so that I will be able to verify that the system actually has the desired effect. It should then be possible to interact with the system without having to rewire the garden. Obviously the system should continue to function (and not lose sensor data) while my PC is switched off.
Side note: As a general principle I am trying not to rely on WiFi at all, but the place where the irrigation is supposed to happen is also actually out of regular WiFi range (unless I were to use additional repeaters). I would have liked to use some powerline based approach but rejected that idee (for now) due to the added costs and engineering complexities. I am not interested in mobile phone based approaches either: First of all the whole point of this exercise is “automation”, i.e. I want to prepare for those moments where I am NOT at home (e.g. when I am on vacation) and where I do not have adequate information regarding which of my plants might be in need of irrigation. Having a phone (or web based) “remote control” for me therefore seems to be but a rather useless gadget. Finally I don’t want to pay for a respective phone contract (that I otherwise do not need) just to irrigate my garden.
This being my first baby-steps in the realm of DIY hardware, I wanted to use something as inexpensive as possible – so that it would not hurt financially regardless of how many electronics components I might fry due to my noobish ignorance.. I therefore decided to base my experiment on some cheap chineese Arduinos. These things cost about a Euro each such that the single most expensive component for all of my devices finally proved to be the plastic case used to package them..
I ended up building three types of devices:
1) soil/air sensors: Equipped with a simple 433MHz transmitter these devices periodically broadcast their measurements, and various power saving techniques make sure that they’ll get through the season on their rechargable 9V battery.
2) RainMaker controller: This controls the irrigation valves (based on configuration and sensor data) and it also buffers and relays (to the PC) the data that comes from the sensors. It uses a somewhat more powerful transceiver for two-way communication with the PC.
3) USB interface: Simple device that is connected to the PC’s USB port. It allows the PC to communicate with the RainMaker device(s) in the field – or any other device that is based on my respective custom communication protocol.
When ordering the parts in small quantities from AliExpress the material costs for the different devices are: 10-12$ per sensor (depending on features), 12$ for the USB interface, 45$ for the RainMaker and another 45$ for the irrigation valves (6-way) assembly.
For the PC side I setup a simple DB to store sensor/irrigation data and I wrote a simple Java GUI used to visualize that data, and to interact with available devices in the field.
So far I successfully completed the dry-testing, and it will be time to test the devices in the field.. where it remains to be seen what additional insights will come of that.
Lessons learnt so far..
- With a bit of help from an electronics savy coach (thank you Markus! 🙂 ) doing a bit of DIY hardware can be a fun experience – even for electronics beginners.
- As soon as a MickeyMouse project gets just a tiny bit “bigger” you may quickly run into the limitations of the inexpensive/smaller Arduino’s hardware (e.g. you want to use a receiver and an additional transceiver, a LCD, a RealtimeClock, some Relais and an EEPROM and you may run out of I/O pins; and even when you circumnavigate that issue (e.g. using I2C and some PISO shift register) you might find that your program – with all the required libs – just barely fits into the available 32k FLASH ROM). Except for very simple devices, optimizing your stuff for size seems to be a part of the Arduino deal. I had last encountered these kind of problems on the C64 some 30 years ago 🙂
- What people seem to call an IDE in the Arduino world can be rather frustrating for anyone who is used to work with a real IDE (aka IDEA IntelliJ, etc). Calling it “AlmostNotepad” would be a more fitting description. Certainly you don’t want to make programming errors in that kind of environment.. ever 😉
- Actually there is a friendly / supportive Arduino community that may considerably help you on your learning curve.
Just a little experiment for how to synchronize visualization of additional data streams with the the playback of WebAudio music: The music samples are generated on the fly using a ScriptProcessor emulating some legacy AtariST. In addition to the stereo output the respective ScriptProcessor also generates three streams containing “playback volume” for the AtariST’s internal soundchip voices:
just for fun a more psychedelic WebGL based rendering of the same data (the WebGL here combines an orbit trap fractal with an inverse distortion, and the “music volume” is used to parameterize the rendering):
are certainly not a good idea if a program is supposed to be portable. Unfortunately that is exactly what ZXTune is using to parse the different binary music files.
“One of the rules of packed structs is that you have to access them directly through the packed struct type. In other words, you can’t, in general, take the address of a packed field and then access the field through that pointer, because the pointer type won’t reflect the packed attribute.” (sunfishcode)
Unfortunately ZXTune used boost::array instances within the various packed structs.. Problem: when methods are invoked on boost::array (or std::array, etc). The ‘this’ argument to the boost::array functions may be misaligned, but the boost::array functions themselves don’t know this.
On CPUs which don’t mind unaligned memory access you may get away within without realizing that there is a problem.. and in this case it was my attempt to cross-compile the program using Emscripten that revealed the issue. Not wanting to rewrite too much of the foreign code I opted for a quickfix: replacing the boost::array with a built-in array fixed the immediate problem…
Naturally a clean implementation should better refrain from depending on unaligned memory access at all… not all the CPUs are as forgiving as Emscripten.
(click on the below image for a live demo).