Fencing

Bit of fun...

We usually put up some Christmas lights on the house - some fairy lights on the metal fencing at the front, but a pain as means a cable out of a window. They are usually just normal fairy lights.

But with my new found expertise in WS2812 style LED strips, and my controllers, I decided to do better.

11m of wooden fence at the front of the house on the road. So let's do this properly. The key point is I have outside power at the end of the fence for the hot tub. So I was able to install, under cover, a 20A 5V power supply.

I then got 4 strings of fairy light style water proof 5V WS2812 LEDs.

I drilled nearly 200 holes, carefully measuring each to be level and evenly spaced. That is surprisingly hard work, LOL. James followed me poking LEDs through the holes. We were both expecting to fall off the damn wall, and James's main concern is I would fall off whilst he was not videoing!

But it is not quite so simple. Just in case you don't know, there are two common issues with LED strips.

Current limit

One issue is max current draw can be too much for power supply. To test you can either work it out, or, simply set all LEDs full white. 200 LEDs is too much for a typical small 5V USB charger plug. Hence the 20A 5V supply.

I actually also did 663 (11m) RGBW LEDs on nice 45 degree extruded trunking with diffusers for the hot tub as well, from same supply. Now that used a lot of current - just one 5m strip is too much for a USB 5V charger when white.

Voltage drop

This is slightly harder to solve. Along the strip the current draw means voltage drops as you go along. Different LEDs need different voltages. First you lose some blue making it yellow, and then some green, making red/pink. And even before that, when white still, you lose some brightness.

So with this 50 LED strip - one strip works. Two strips work but losing brightness at end. Three strips means going distinctly yellow at the end. I wanted four strips!

The solution

The solution is power feed in - the strips even have extra tails for power as well as the three wires for power and data. You feed in extra power at each strip end, so for my 4 strips I feed in at 5 points.

But how do you feed in power? In some cases you could simply power your longer strip at both ends and not have enough drop to the middle to notice. But I don't have power at the other end.

But actually it is possible to feed in even with just power from one end. The reason is the resistance of the wires, these are classic Chinesium™thin wire. If you actually have some thick good quality copper wire you can run and extra power lead the whole length and feed in at each strip end. This is what is in the WAGO boxes in the image.

Merry New year!

P.S. my son sells the controllers and stuff, https://hiwtsi.uk/

Update: Measuring resistance on the 50 LED strip power lines showed 1Ω but the leads were 0.1Ω, so 0.9Ω. A similar length of copper wire registered 0.4Ω, so 0.3Ω, so ⅓ of the resistance.

James did a video :-)

Playing cards

One of the fun diversions I have had in my time was making playing cards. I did a whole chapter in my biography on this.

My playing card design site https://www.me.uk/cards allows you to make a wide variety of cards. It is a fun little system I set up long ago.

However it has come up lately for a few reasons.

For a start I made some cards for the pub, on Amazon. Please buy some.

But also some error reports I had - some edge cases made bad cards. And making the cards for the pub meant I wanted custom card backs which it did not allow.

So I have updated. New features...

• Fixed a bug making some size cards mess up court cards.
• Upload custom artwork (PDF) for backs.
• Upload custom artwork (PDF) for jokers.
• Maze and arrows backs are more random, each deck is different (obvious all cards the same back in each deck - but every deck we make is unique).
• Tidied the options to be clearer.
• Added an option for a second set of aces to be included.

The last point was one I pondered. We make some unique decks, with an "11", or a "0" or "1" card, which is unusual. But actually what may sell better is a deck with a second set of aces, to have, well, "up your sleeve". So why not.

I have added custom ace of spades now too.

Playing with microphones

The latest LED board designs have included a TDK PDM I2S microphone - the idea was to make sound reactive LED strips.

It is tiny (3.5mm x 2.65mm x 0.98mm) and cheap(ish) $0.76. But it is digital.

Now, this was a can of worms for which I was ill prepared. I2S is not like I2C, it seems, and there are a number of ways of doing it.

This particular device is PDM based, so has just a clock and data line - the clock to the microphone, and data comes back. The concept is that you have two of these, one set for right and one set for left channel, on the same clock and data. The left and right are on the rising or falling edge of the clock. It is fixed 16 bit per channel, and works at a specific (wide) range of frequencies of clock and hence sample rates.

But other I2S devices work in a different way, with extra pins.

I went through some stages for this...

• Simple analogue microphone (cheaper) - realised that was a daft idea.
• This mic, set to right channel, and it did not work.
• This mic, set to left channel, and it did not work.

What did not work was using WLED, a common LED driver package which works nicely on our LED modules. We realised it wanted I2S, and then realised it only did PDM on left channel, hence the different iterations, but no joy. Hopefully we can get some feedback in to that project so it does work as TDK is likely a good brand. It is amazing how sensitive it is.

My guess is it is not clocking at a rate the device will handle when using WLED, sadly.

So time to do it myself, as always!

I coded I2S as per ESP32 library, and, well, it worked, I got raw data. I could even tell it I want mono only and which channel. Amusingly using the wrong channel works too as the data floats for the other channel - though I picked up some high frequency noise that way. So best to set it correctly (I nearly said "right").

The next step was a simple FFT function - I have not gone for anything fancy or uber efficient, as it keeps up well enough, especially when I went for some simple oversampling on its input. I cannot clock the microphone as slow as I would like for normal working. But even at 30k samples/sec I could keep up with FFT 30 times a second, just, all done with floats. These ESP32s are impressive.

I have made the working audio range configurable, and averaged the FFT down to a small number of bins (24) which are log of frequency to fit better with musical scales (options for linear).

This has led to some simple audio responsive LED schemes - a simple brightness based audio spectograph - smoothing from 24 bins to however many LEDs you have in a chain. And also a simple overall volume based bar graph, and one that is RGB for three levels of sound.

Lessons learned.

Some automatic audio gain was pretty simple.

The next lesson was the log scale on frequency. That was not hard. But made an option.

I reduced the full FFT result to a small number of bands, 24.

Also I needed to make it do a peak level per band and damping (configurable) to give anything that looks nice.

The frequency range is configurable, but log based causes gaps if you go too low, so ended up 100Hz to 4kHz for now, log spread over the 24 bins from a sample rate at a typical 25 blocks/sec and actually sampling over 50-60ks/s down graded (average of oversample) to 12-15ks/s for the FFT giving results to 6-7.5kHz which is more than enough for the bands I am using.

I also ended up making the samples fit the LED update rate synchronously, typically 25 or 30 per second (configurable) to avoid any extra jitter/aliasing effects.

Oh, and when using RGBW, with a colour band for spectograph, before automatic gain kicks in you can have over 1 on a band, so I made that push in to turning on the W, which worked well.

The result

Update: I have now tweaked and got cleaner response across the range, putting 50Hz to 6.4kHz in to 42 log based frequency bins. See youtube for frequency tests.

Now for use with a live band at the pub!

And HIWTSI (my son's business) is doing LED system installs now.

One Touch Switching

It has been some weeks since One Touch Switching was fully live.

TOTSCO say over 100,000 switch orders now, so it is making good progress, well, in principle.

In practice a lot is working, and in terms of volume, with the key players, as well as the likes of A&A, all working reasonably well now, switches are happening, both ways. We are seeing things working both ways and correct billing as well, which is good.

But there are still some challenges.

• Whilst I cannot go in to details, even the big players are still facing some issues, mostly small issues but some bigger, and some with workarounds for now, and for which they are rolling out as updates. Daily calls continue with industry (yes, some I have taken from the pub).
• A lot of smaller players are catching up, but many face the challenges of the huge holes in the specifications. These are still frozen and so grey areas, errors, and contradictions abound. Only once they are live with other ISPs are the issues apparent. In at least one case we have to ask how a CP managed to be live as they had some fundamental errors (like SOR not being a UUID) that should not have passed even the very poor testing TOTSCO claim to do. Even larger CPs are not entirely agreed on some field specifications because of poor wording in one of the better parts of the spec, but are working on it. The daily calls help, but only happen with the early/bigger CPs.
• There is now a formal process for inter-CP communications to resolve such issues, but not everyone is on it yet. There is a fallback process as well. So things are happening, and some of these issues with smaller CPs are being fixed as well.

But even now, even this weekend, I have seen incorrect messages and errors. I have reported, of course, and it may be these end up as defects on the daily calls, we will see.

I worry what will happen when the daily calls stop - reporting an issue to a CP may mean they ignore it rather than spend resource investigating, fixing, and deploying. At present CPs have that resource assigned, but they will not forever.

What next? Well we keep at it.

The next big step I can only hope for is an unfrozen spec and a lot of clarifications and updates. It will be interesting to see how that process happens, and how we can be involved. I have a lot to say on clarifying the specifications and I hope I can be involved in making it happen. But every change will need a lot of agreement, and even some changes by all CPs in some cases. For now, there are some silly compromises like all strings max 256 characters (which resulted from a global update to a Swagger definition system, rather than any informed debate or formal specification change, and is annoying as tinytext in mariadb is 255 characters not 256). Even so, some agreement on even the vague magnitude of things like correlationID is a good start. I suspect, in practice, that one may get defined as smaller, like 64 characters. In hindsight it should have been a UUID, unique per message, but too late to do that now. The problem is the smaller/newer CPs are not in on that discussion, so don't know. Big CPs guessed at 36 characters (UUID size), 56 characters, 64 characters, and so on, as there simply was nothing in the spec, but most had to set something in their code. We changed our handling within the first few days as we understood how broken the spec was, and now handle any size (well, megabytes) but other CPs don't, and we have limits on a load of other strings anyway.

For now, I have every message we receive, and every message we send, run through my NOTSCO checker and reported to me. I feel it is only fair to test us as well. Over time it will only be problem messages that I need to monitor. It has actually highlighted some issues in what we were sending (where customers manually type an address, mainly - I have added more checks now). But monitoring real life message has also meant updates to our checking in the live system, and updates to my NOTSCO tools.

My latest changes include actually using the longer agreed size of correlationID to ensure we tag the message type as a (small) suffix on a UUID, so that we can quickly (pre-database connect/check) validate messages we get back are sensible and reject them. Why? Well one small CP is sending nonsensical replies to replies, or reusing correlationIDs from previous messages with different messages, both of which we can now pick up in milliseconds, and cleanly reject. It looks like they are working on it, but no actual communications back to us, which is a shame - we're happy to help and advise if only they would talk to us.

Overall - OTS is happening and mostly working, so do try it when you want to switch telco.

Pubbing it

I used to go to the pub once or twice a week, a few years ago, not so much since COVID.

What is weird is finding myself going to the pub almost every afternoon now.

OK, well, morning as well, even if that is installing a PIR, or Access Point, to CCTV camera, or door contact, or just some signs. But then in afternoon just go see how it is going - and people watching...

Today I jogged in to meet an unexpected BT engineer!

This is a bit new to me.

Another pub

If you have followed me on mastodon you will know what is happening.

Some of you may know I purchased a former pub for my home in Wales nearly 4 years ago. Not a pub now. Settling down in Wales, as are many of my children, and grandchildren (half of which are Welsh!).

But now, two of my friends have decided they would like to actually run a pub!

There is a saying: "How to make a small fortune running a pub? Start with a large fortune and work at it until it is a small fortune".

But that does not mean it is impossible, and it seems sometimes that the whole industry is run on the edge of solvency somehow. I may be wrong, and obviously there are pub+hotel and other combinations that work well, but the way some pubs are run is, err, interesting.

I have got myself involved in helping them in a lot of ways now. It started as just "we'll sort internet", but doing more and more. I'm spending most days there helping out right now.

It will be very interesting to see if they can make it work. They are organised, and experienced in related businesses. They are hard working. They take customer service seriously. I think this can work.

Lots of little things, like a defibrillator on the pub - I'm sponsoring that (well A&A are) - one of my staff died suddenly and that scares the shit out of me, more places having these will definitely save lives. And if it saves one life it is worth every penny.

And, of course, the WiFi has IPv6.

They are going for pub + cafe, with decent coffee, and choice of milk and so on, like many other "coffee shops", but also with early opening for coffee and breakfast. Do the coffee right and it could make as much as the alcohol, who knows?

But making sure they are starting it out well from a tech side is my project. They are sorting refurbishing and redecorating, and some serious cleaning. I am sorting decent WiFi/internet, and CCTV and a few other things. Even to the extent of properly registered with ICO and proper published privacy policy. I hope I can provide input on the way some of us would like a pub run. But it is their pub, not mine.

But for now - grand opening end of next week before Abergavenny food festival (only drinks for now), and then finish the refurbishment and start food as well.

I am, however, damn impressed with the WiFi we have put in. It does not just cover the front, but the whole of the Brewery Yard car park and even out the to A40. Some opportunities for take away coffee to the market traders maybe, with the working WiFi connected credit card machine... We'll see how it goes.

I'll post more on opening dates, and so on...

OTS correlation ID

It is complicated to report this as some details would be covered by ramp up rules, where I cannot provide details, but now we are in live on One Touch Switching I think I can report. Even so I will not name CPs for now. This is more about the process, and specification and the fiasco.

To be very fair the main author of the specifications is someone that I feel a lot of sympathy for, under pressure, and then under a change freeze he would not have agreed. He did his best and none of this is a dig at him or his employer. Much more a dig at the process.

Weirdly correlationIDs have turned out to be a big issue, and continue to be so.

What is a correlationID?

Basically one of the message fields in One Touch Switching is called a correlationID

The big issue is the vague specification. It is a field the sender of a message sets so they can correlate the response.

Hindsight

To be clear, in hindsight, and what I have said, is correlationIDs should be per message unique and a UUID. Simple. If the spec had said that a lot of pain and hassle would have been avoided.

Problems?

The problems are various...

• It is not a defined format
• It is not a defined maximum length
• It is not well defined when and how it is unique, or not

TOTSCO 66

So the issue is that some CPs assumed it would be per CP per message unique, and so used it to identify (and ignore duplicates). Indeed a notice from TOTSCO suggested it is used to de-duplicate messages.

A real issue is "why duplicates" which is another issue - they have to be failing to respond in 3 seconds for that, and maybe that is what they should have fixed.

There are also a lot of cases where a duplicate is not an issue, if done right.

But TOTSCO 66 said you can de-duplicate based on per CP correlationIDs being unique per message.

TOSTCO 67

The next notice back peddled a lot, and I was instrumental in raising this I think. All because the specification was so vague. The new recommendation was two fold (a) don't de-duplicate on correlationID, and (b) don't send duplicate correlationIDs. A pragmatic approach without direct blame either side.

Indeed one idea was, if you use correlationID as a more "overall message flow" ID, append or prepend something to each message so they ends up unique.

So CPs are, indeed, doing both, yay! We have all seen a lot of work making this happen, and well done to all the CPs doing this.

To clarify we went through something like three iterations to get this sensible on our systems.

Length

Oh, did the specification say how long a correlationID could be? No. It did not. Why would you say that?

Well, maybe it did, sort of, TOTSCO link to some schema thing (swagger?!) which was updated after the frozen spec and the latest version of that says 256 characters. That is mental long, and I have no clue if 256 characters or 256 bytes (they are different in the UTF8 world of JSON). Just to say, A&A can handle any length up to mega bytes, if needed.

Turns out TOTSCO had limits on what they would handle, as this is a message envelope thing. They were ignoring, and not apparently reject cleanly, if too long. I have not tested with 256 x big unicode characters, yet!

But we have a big CP that would not handle more than 64 characters, but sorted that before 12th, well done, I won't say who. It was a very reasonable choice for them, and I understand it. But well done moving to 256 characters, or bytes, in time.

We now have another big CP that would not handle more than 50 characters. Not yet sorted, but will be soon.

Why such long correlationIDs? Well BECAUSE of TOTSCO 67 notice, CPs using a 36 character UUID and adding a timestamp. That just pushes over 50. And to be honest 50 was also a reasonable design choice.

So 256 characters, is that OK? Guess what, the tinytext type in mariadb is 255 characters, FFS! If I had to make a silly long limit I would have said 255 not 256, really.

Ping pong

One of the mistakes we made at the very start, for a day or so, was assuming correlationIDs were ping ponged over the switch process (match, order, update, trigger). I had fields in the database to do this and code to do it (they were tinytext).

Why did I assume this? Well the specification did not say, but the test cases did, they had correlationIDs on an OrderRequest following on from the MatchRequest. They looked a lot like they should ping pong over the process.

Well I worked it out, but did every small CP that is live today?

The answer is no, they have not, and at least one small CP (I feel sorry for them) very carefully followed the spec, and the examples in the test schedule, and did this, like we did.

They will not work with almost any of the other CPs now live. They have to make major changes, now, when live. Really sorry for them.

Helping?

Seriously, we all need to work together. We have a test system that can help these new small CPs, and I am happy to help. https://notsco.co.uk/