I thought I'd update a bit on some of the work at A&A over the next month...
Talk Talk backhaul
As I mentioned recently, there have been issues. It is only some parts of their network, but it is not what we want. The good news is progress is being made on their new network and they are working closely with us to test before it can go live. I am very keen for customers to be switched over to this new network before Christmas, well, as soon as we can. The issues have been quite complex and Juniper engineers are on the case with Talk Talk. On our side we are helping in any way we can with test lines and packet dumps, and even making LNS code changes to test ideas and work arounds. This is one advantage of making our own routers with FireBrick...
BT backhaul
The BT backhaul is no issue, but we are extending the capacity for the future. This is making a lot of progress, though today I heard BT have a delay as they don't have any patch cables. We literally pointed them to www.shop.bt.com and suggested they buy some! It will take a while to have the new links up and running as there is a lot more work than just a few patch cables. But that should be soon, and before Christmas. This work is what is holding off some of the new tariff options we are working on.
New SVG graphs
We are updating our systems to some new SVG based usage graphs, as I have blogged. That is going to take a few weeks to fully roll out and set up user options and fine tune. We are keen to ensure customers can see how well their service works, with no sugar coating. The new graphs provide more detail than before, and should be clearer.
Christmas cover
We will, of course, ensure cover for normal non bank holidays over Christmas, but we should be able to confirm the cover for other days as we normally work Saturdays as well. I'll make sure we do some sort of announcement. This year makes it a bit simpler as no arguments with BT as to whether a 25th Dec on a Saturday or a Sunday is a bank holiday or not (it is not). As always, there will be some informal out of hours support and emergency support for major issues. Those few customers still on our old "units" tariffs still get the Christmas week as off-peak as well as bank and public holidays.
2017-11-30
Trademark
As you know, I have a twitter account - and was even described as a prolific tweeter by someone. OK, that is a bit much, only around 2500 followers, but none the less, I tweet, therefore I am.
My twitter handle is @TheRealRevK but the name is RevK which I use extensively since I signed up as a reverend with ulc.org many years ago. I use it in a lot of places, and even on this blog. ULC don't require a belief in a deity before ordaining you :-)
I do not have @revk as someone else has it, but it is a dormant, never used, twitter account.
Contacting twitter is almost impossible, but someone said they do have a policy on trademarks, and they have a support ticket help page for that and a form.
So, after literally a few minutes on-line, if that, I have paid £170 to the UK Intellectual Property Office to apply for a registered trademark, personally, for blogging, as the text "revk". I see no others when searching, so that stands a good chance or being accepted. I'll know soon.
That then means I can actually defend against other users of "revk" and maybe get the twitter handle. I may not manage it, after all even Donald Trump still has "real" in his handle. If I succeed where he failed I will be well chuffed. More on this story as it happens.
My twitter handle is @TheRealRevK but the name is RevK which I use extensively since I signed up as a reverend with ulc.org many years ago. I use it in a lot of places, and even on this blog. ULC don't require a belief in a deity before ordaining you :-)
I do not have @revk as someone else has it, but it is a dormant, never used, twitter account.
Contacting twitter is almost impossible, but someone said they do have a policy on trademarks, and they have a support ticket help page for that and a form.
So, after literally a few minutes on-line, if that, I have paid £170 to the UK Intellectual Property Office to apply for a registered trademark, personally, for blogging, as the text "revk". I see no others when searching, so that stands a good chance or being accepted. I'll know soon.
That then means I can actually defend against other users of "revk" and maybe get the twitter handle. I may not manage it, after all even Donald Trump still has "real" in his handle. If I succeed where he failed I will be well chuffed. More on this story as it happens.
Today's recipe: Grilled cheese on toast
I do feel I should point out that (a) my wife does feed me, these are just snacks I make, and (b) I am not planning for A&A to open a restaurant or any food business.
The first step in this recipe is bread, obviously, but the choice of bread is perhaps not quite so obvious. I find that over the years I have preferred a particular type of bread, some times for years, and then decide I like something different. Brown, granery, white, all sorts. I'll even go through phases of not wanting any toast for months, very odd. The latest bread I particularly like is corn bread which I can get from Tesco.
Corn bread is a bit odd. I tried it without toasting it once, and it is a disaster, all stodgy. But toasted it is lovely. So start by toasting it...
Then butter...
Then marmite...
Then comes the cheese. Now, again, choice of cheese matters. I do rather like the smoked applewood with the paprika dusting...
I usually do this finely grated, but as I have some pre sliced cheese, I may as well use that. Have you tried grating pre-sliced cheese? Talk about lazy. At least I sliced my own bread.
The trick at this point is a little seasoning, and that is perhaps what makes this an actual recipe, for some stretched definition of the word...
Jamaican Jerk is great for cheese on toast, but seriously, how the hell does anything like this have an expiry date ?!?
Anyway, sprinkle on the cheese...
And grill until melted nicely.
Delicious.
The first step in this recipe is bread, obviously, but the choice of bread is perhaps not quite so obvious. I find that over the years I have preferred a particular type of bread, some times for years, and then decide I like something different. Brown, granery, white, all sorts. I'll even go through phases of not wanting any toast for months, very odd. The latest bread I particularly like is corn bread which I can get from Tesco.
Corn bread is a bit odd. I tried it without toasting it once, and it is a disaster, all stodgy. But toasted it is lovely. So start by toasting it...
Then butter...
Then marmite...
Then comes the cheese. Now, again, choice of cheese matters. I do rather like the smoked applewood with the paprika dusting...
I usually do this finely grated, but as I have some pre sliced cheese, I may as well use that. Have you tried grating pre-sliced cheese? Talk about lazy. At least I sliced my own bread.
The trick at this point is a little seasoning, and that is perhaps what makes this an actual recipe, for some stretched definition of the word...
Jamaican Jerk is great for cheese on toast, but seriously, how the hell does anything like this have an expiry date ?!?
Anyway, sprinkle on the cheese...
And grill until melted nicely.
Delicious.
2017-11-29
Today's recipe: Chicken soup
As you may imagine, my culinary skills are limited, so for today's recipe, chicken soup, one starts with, you guessed it, a tin of chicken soup...
Which you microwave for 2 minutes...
Then add some bacon bits...
Then some chilli flakes...
Some salt...
And, of course, a dob of marmite...
And stir...
Delicious...
Which you microwave for 2 minutes...
Then add some bacon bits...
Then some chilli flakes...
Some salt...
And, of course, a dob of marmite...
And stir...
Delicious...
Tracing a bitmap
I explained in my blog on SVG how I needed to trace a bitmap for a QR code in order to make an SVG version.
To me, this seems very obvious, but it is interesting that people do have different algorithms for doing this. So I thought it worth documenting how I did it.
The objective is to make an SVG, which uses a path to draw around something and fill it with a colour, from a bitmap. A bitmap has each pixel that is a colour on a grid. It is very obvious how a bitmap works when looking at the QR code as it is clearly made of squares of black and white.
Simplistic
This obvious and simplistic approach is make an SVG rectangle for each pixel. That makes for a large SVG file, hence the idea of tracing around multiple pixels of the same colour and using a path.
Not just black and white
The first thing I considered was that this could be more generic than just black and white. We already use an image library to generate a bitmap using indexed colours. This means we have a grid of bytes, each of which has a value 0 to 255, each representing one colour. A separate table says which colour is actually used for each number. It is a paint by numbers system.
I already use this to generate the existing PNG images, including the QR code. But why not make the tracing function a generic SVG output for any indexed bitmap we make.
This basically means tracing each colour, one after the other. However, the principle is just the same for black and white images.
Background colour
Even with an indexed colour bitmap there is often the notion of a background colour. So my code starts by checking if one has been specified and making a simple rectangle of that colour for the whole bitmap.
With the QR code this is simple, it is white. Not a lot of people know that QR codes actually require 3 pixels of white all the way around. You often see them with one pixel, or even in some cases with none (krispy kremes boxes?).
Sometimes the background colour is also transparent in which case you don't plot it at all. Obviously a simple check to see if any pixels are the background colour, and if none, then don't bother either.
In any case, if there is a background, you don't include that colour in the tracing as it is already handled.
Just black
Obviously for the QR code it is simple. The indexed colours are just 0 (white) and 1 (black), with 0 as the background colour. So we do a white rectangle, and then a path for the black. Where there are multiple colours we just trace each (non background) colour one at a time.
Directions
As part of the walking around the image tracing it, we have to consider facing a direction. Imagine you are standing on a square on a chess board, you can be facing one of 4 ways, to consider the 4 sides of the square on which you stand...
When working on a direction, some times we have to add or subtract, and we do so using modulo arithmetic, so adding 1 to 3 becomes 0. As such adding 1 is turning right and subtracting 1 is turning left.
In the code, a simple dx and dy table can be used to hold the offset for each direction.
Repetition
As we scan the image to find areas to trace around, we could easily find we are going over the same thing more than once. To avoid this we also create a byte array, one per pixel, in which we record the edge of that pixel which we have already traced. This is one bit per direction so actually only 4 bits used in each byte. This stops us accidentally trying to trace any edge of any pixel more than once. We only record the edge of the pixel we are in, not the other side, i.e. the opposite edge on the adjacent pixel, as that will be recorded when we trace its colour. Obviously if the other colour is the background, as with out QR codes, then that does not get traced, but if it was in fact another colour to trace, we don't want to record that edge as having been traced yet.
Over the edge
Obviously, as part of this, we need to look at the colour of a pixel, so we make a function that takes an x and y co-ordinate and returns the colour. The indexed colours are 0 to 255. However, we allow full integers for x, y, and the result. This means we can look over the edge of the bitmap, either negative x, or y, or values beyond the width and height of the image, and these can return -1 for the colour which will match no colour actually in the image.
What this means is we can look at the pixel in any direction and see that it is a different colour, so it is the edge of what we are tracing. We do not have to have a special case for looking in direction 3 when on the left of the image, we simply see that as colour -1, so clearly the edge of what we are tracing.
Finding areas to trace
The first thing to do is scan the image for the colour we are tracing. Remember, we are tracing one colour at a time, and not bothering to trace the background colour at all. For QR codes, that means we only trace black.
Two faced
Actually, it is slightly simpler! I said we check 0, 1, 2 and 3 to find an edge to trace, but in practice we can only see 0 or 2.
It may seem obvious actually that you should only ever see 0. After all, the pixel above must be a different colour. If it is the same colour then we already did it, and so the whole block of pixels we are in has already been traced, and so if direction 0 is the same colour, any edge on 1, 2, or 3 must have been traced as part of that block, surely.
Well, actually no, there is an edge case (:-) if the block of pixels we have already traced, i.e. direction 0 is the same colour, but it fully encompasses pixels of a different colour.
Whilst the outside of this black block has been traced, this is an edge to an interior area, and so has not been, and it is direction 2.
So why could we not have direction 1 or 3? Well if it was 1, that means the interior area in question would have been found one row further up already with a black pixel facing down (direction 2) already. Similarly, if the interior pixel was on the left (direction 3) it would have been seen facing down (direction 2) from a previous row.
So actually there are only two possible ways would could be facing to start, 0 and 2. Not that it matters, but 0 is exterior and 2 is interior.
Doing the trace
Having found a starting point, you then do the trace. The direction you are facing is the edge you are going to draw, for each edge you only need to plot one point, and you make up a set of points that are going to be a closed filled area. In SVG you use M x y for the start and L x y for a line, so you need a flag to know you are doing the first point and use the M.
But which point is it? You are looking at an edge, and it does not really matter if you pick the left or right point on that edge in the direction you are facing, as long as you are consistent. In this example I pick the one on the right (considering the direction I am facing).
You mark this edge has having been done in your byte array of edges. And you add this point to the list of points for this trace.
You then move on, and this is where you have to walk around the area you are tracing. So we have to make a decision.
There are three possible choices of how you move...
Firstly, can you turn left around a corner? This means moving to a pixel that is one space right from the direction you are facing (d+1), and then one place left of that (d). To decide if you can move there you simple check if that pixel is the same colour. If so, then move there, and also turn left (d-1). This leaves you facing the same pixel you started facing, so you know it is a different colour already and so don't have to check.
If you can't do that, you may be able to move right. This is only if you cannot do the above move. To do this you simply check if the pixel to your right (d+1) is the same colour. If it is, you move there, but do not change your direction (d). You end up facing the pixel you could not move to in the above case, so again, you know you are facing a pixel of a different colour still.
Finally, if that does not work, you simply turn right, this means staying in the same place by changing to (d+1) direction. Again, this leaves you facing the pixel you could not move to in the above case, so again, you know you are facing a pixel of a different colour still.
When to stop
You keep going, adding a new point, and marking an edge as done, until you find you are facing an edge that is already done. That means you have got back to the start. Then you move on to the next pixel in your scanning to find the next area to trace.
Closing the path
This does not actually result in plotting the end point, as you stop when you get back to the start, but that is fine as the SVG full does not expect the path to be closed for a fill. If you need to, because you want to also trace the outline, for colour bleed or some such, then you can used the close path command (z).
Enclosed areas
The good news is that this approach does not have to consider enclosed areas. We actually know if it is an enclosed area by the starting direction being 2 and not 0, but that does not actually matter. This tracing will always create a clockwise trace for an exterior path and a counter clockwise path for an interior path. The fill logic used by SVG means the inner path cancels our the outer path and creates exactly the effect you want.
Colours butting up
There is one possible issue when tracing multiple colours. The above will create the traces for each colour, but the areas of one colour will touch the areas of another. This should not be a problem in SVG or other contexts, but could be - i.e. depending on how things are rendered it could lead to an infinitesimally small gap between them - perhaps if the result is rotated or manipulated in some way.
To avoid this, we could change the trace logic to consider any colour of a higher number to be the same colour, and we know that the higher colour will be traced on top of this colour. If we do this we will have to clear the byte array of edges before each colour as we'll use the same edges.
Sadly this creates the same issue where we have the edge of two colours on top of each other, and any misalignment due to processing could leak one colour overlapping the other by an infinitesimally small amount.
So maybe not an issue worth bothering about.
Optimising the result
Another important point is not simply to use M and L for SVG. Doing so means absolute co-ordinates and a point for each end of each edge.
Instead, I simply made some functions that use M at the start and then relative co-ordinates using l, h, and v. The numbers are smaller, and using h (horizontal) and v (vertical) means only one value. In practice l is not used, but they are generic SVG path functions that could be used in other places.
The optimisation obviously has to consider that we have multiple points in a row and reduce to a simple line. In this case it was only necessary to consider multiple points either horizontally or vertically which reduce to a single h or v command.
The other optimisation is to use the point on the left of the edge you are facing when doing an exterior path, but the point on the right when doing an interior path. This means you always start and end on a corner, making the optimisation slightly easier.
Obviously if you want an SVG where the QR code pixels are bigger, as is often the case, then simply enclosing the entire thing in a g (group) with a scale is simplest and keeps the numbers in the paths smaller.
To me, this seems very obvious, but it is interesting that people do have different algorithms for doing this. So I thought it worth documenting how I did it.
The objective is to make an SVG, which uses a path to draw around something and fill it with a colour, from a bitmap. A bitmap has each pixel that is a colour on a grid. It is very obvious how a bitmap works when looking at the QR code as it is clearly made of squares of black and white.
Simplistic
This obvious and simplistic approach is make an SVG rectangle for each pixel. That makes for a large SVG file, hence the idea of tracing around multiple pixels of the same colour and using a path.
Not just black and white
The first thing I considered was that this could be more generic than just black and white. We already use an image library to generate a bitmap using indexed colours. This means we have a grid of bytes, each of which has a value 0 to 255, each representing one colour. A separate table says which colour is actually used for each number. It is a paint by numbers system.
I already use this to generate the existing PNG images, including the QR code. But why not make the tracing function a generic SVG output for any indexed bitmap we make.
This basically means tracing each colour, one after the other. However, the principle is just the same for black and white images.
Background colour
Even with an indexed colour bitmap there is often the notion of a background colour. So my code starts by checking if one has been specified and making a simple rectangle of that colour for the whole bitmap.
With the QR code this is simple, it is white. Not a lot of people know that QR codes actually require 3 pixels of white all the way around. You often see them with one pixel, or even in some cases with none (krispy kremes boxes?).
Sometimes the background colour is also transparent in which case you don't plot it at all. Obviously a simple check to see if any pixels are the background colour, and if none, then don't bother either.
In any case, if there is a background, you don't include that colour in the tracing as it is already handled.
Just black
Obviously for the QR code it is simple. The indexed colours are just 0 (white) and 1 (black), with 0 as the background colour. So we do a white rectangle, and then a path for the black. Where there are multiple colours we just trace each (non background) colour one at a time.
Directions
As part of the walking around the image tracing it, we have to consider facing a direction. Imagine you are standing on a square on a chess board, you can be facing one of 4 ways, to consider the 4 sides of the square on which you stand...
When working on a direction, some times we have to add or subtract, and we do so using modulo arithmetic, so adding 1 to 3 becomes 0. As such adding 1 is turning right and subtracting 1 is turning left.
In the code, a simple dx and dy table can be used to hold the offset for each direction.
Repetition
As we scan the image to find areas to trace around, we could easily find we are going over the same thing more than once. To avoid this we also create a byte array, one per pixel, in which we record the edge of that pixel which we have already traced. This is one bit per direction so actually only 4 bits used in each byte. This stops us accidentally trying to trace any edge of any pixel more than once. We only record the edge of the pixel we are in, not the other side, i.e. the opposite edge on the adjacent pixel, as that will be recorded when we trace its colour. Obviously if the other colour is the background, as with out QR codes, then that does not get traced, but if it was in fact another colour to trace, we don't want to record that edge as having been traced yet.
Over the edge
Obviously, as part of this, we need to look at the colour of a pixel, so we make a function that takes an x and y co-ordinate and returns the colour. The indexed colours are 0 to 255. However, we allow full integers for x, y, and the result. This means we can look over the edge of the bitmap, either negative x, or y, or values beyond the width and height of the image, and these can return -1 for the colour which will match no colour actually in the image.
What this means is we can look at the pixel in any direction and see that it is a different colour, so it is the edge of what we are tracing. We do not have to have a special case for looking in direction 3 when on the left of the image, we simply see that as colour -1, so clearly the edge of what we are tracing.
Finding areas to trace
The first thing to do is scan the image for the colour we are tracing. Remember, we are tracing one colour at a time, and not bothering to trace the background colour at all. For QR codes, that means we only trace black.
- We scan from top to bottom and left to right in a simple loop looking first for a pixel of the colour we are tracing.
- We then consider what edge of that pixel we want to trace, looking in turn at direction 0, 1, 2, 3 and looking for a pixel in that direction which is not the same colour, so an edge of the block of pixels of this colour. We also disregard any directions where we have already done that edge of this pixel.
- If we find no directions that match that rule, then this is not a pixel to trace. It is either one we have done already, or one that is surrounded by pixels of the same colour so does not need tracing.
- If we do find a direction, then that is our starting point for walking around a set of pixels of this colour, doing the trace.
Actually, it is slightly simpler! I said we check 0, 1, 2 and 3 to find an edge to trace, but in practice we can only see 0 or 2.
It may seem obvious actually that you should only ever see 0. After all, the pixel above must be a different colour. If it is the same colour then we already did it, and so the whole block of pixels we are in has already been traced, and so if direction 0 is the same colour, any edge on 1, 2, or 3 must have been traced as part of that block, surely.
Well, actually no, there is an edge case (:-) if the block of pixels we have already traced, i.e. direction 0 is the same colour, but it fully encompasses pixels of a different colour.
Whilst the outside of this black block has been traced, this is an edge to an interior area, and so has not been, and it is direction 2.
So why could we not have direction 1 or 3? Well if it was 1, that means the interior area in question would have been found one row further up already with a black pixel facing down (direction 2) already. Similarly, if the interior pixel was on the left (direction 3) it would have been seen facing down (direction 2) from a previous row.
So actually there are only two possible ways would could be facing to start, 0 and 2. Not that it matters, but 0 is exterior and 2 is interior.
Doing the trace
Having found a starting point, you then do the trace. The direction you are facing is the edge you are going to draw, for each edge you only need to plot one point, and you make up a set of points that are going to be a closed filled area. In SVG you use M x y for the start and L x y for a line, so you need a flag to know you are doing the first point and use the M.
But which point is it? You are looking at an edge, and it does not really matter if you pick the left or right point on that edge in the direction you are facing, as long as you are consistent. In this example I pick the one on the right (considering the direction I am facing).
You mark this edge has having been done in your byte array of edges. And you add this point to the list of points for this trace.
You then move on, and this is where you have to walk around the area you are tracing. So we have to make a decision.
There are three possible choices of how you move...
Firstly, can you turn left around a corner? This means moving to a pixel that is one space right from the direction you are facing (d+1), and then one place left of that (d). To decide if you can move there you simple check if that pixel is the same colour. If so, then move there, and also turn left (d-1). This leaves you facing the same pixel you started facing, so you know it is a different colour already and so don't have to check.
If you can't do that, you may be able to move right. This is only if you cannot do the above move. To do this you simply check if the pixel to your right (d+1) is the same colour. If it is, you move there, but do not change your direction (d). You end up facing the pixel you could not move to in the above case, so again, you know you are facing a pixel of a different colour still.
Finally, if that does not work, you simply turn right, this means staying in the same place by changing to (d+1) direction. Again, this leaves you facing the pixel you could not move to in the above case, so again, you know you are facing a pixel of a different colour still.
When to stop
You keep going, adding a new point, and marking an edge as done, until you find you are facing an edge that is already done. That means you have got back to the start. Then you move on to the next pixel in your scanning to find the next area to trace.
Closing the path
This does not actually result in plotting the end point, as you stop when you get back to the start, but that is fine as the SVG full does not expect the path to be closed for a fill. If you need to, because you want to also trace the outline, for colour bleed or some such, then you can used the close path command (z).
Enclosed areas
The good news is that this approach does not have to consider enclosed areas. We actually know if it is an enclosed area by the starting direction being 2 and not 0, but that does not actually matter. This tracing will always create a clockwise trace for an exterior path and a counter clockwise path for an interior path. The fill logic used by SVG means the inner path cancels our the outer path and creates exactly the effect you want.
Colours butting up
There is one possible issue when tracing multiple colours. The above will create the traces for each colour, but the areas of one colour will touch the areas of another. This should not be a problem in SVG or other contexts, but could be - i.e. depending on how things are rendered it could lead to an infinitesimally small gap between them - perhaps if the result is rotated or manipulated in some way.
To avoid this, we could change the trace logic to consider any colour of a higher number to be the same colour, and we know that the higher colour will be traced on top of this colour. If we do this we will have to clear the byte array of edges before each colour as we'll use the same edges.
Sadly this creates the same issue where we have the edge of two colours on top of each other, and any misalignment due to processing could leak one colour overlapping the other by an infinitesimally small amount.
So maybe not an issue worth bothering about.
Optimising the result
Another important point is not simply to use M and L for SVG. Doing so means absolute co-ordinates and a point for each end of each edge.
Instead, I simply made some functions that use M at the start and then relative co-ordinates using l, h, and v. The numbers are smaller, and using h (horizontal) and v (vertical) means only one value. In practice l is not used, but they are generic SVG path functions that could be used in other places.
The optimisation obviously has to consider that we have multiple points in a row and reduce to a simple line. In this case it was only necessary to consider multiple points either horizontally or vertically which reduce to a single h or v command.
The other optimisation is to use the point on the left of the edge you are facing when doing an exterior path, but the point on the right when doing an interior path. This means you always start and end on a corner, making the optimisation slightly easier.
Obviously if you want an SVG where the QR code pixels are bigger, as is often the case, then simply enclosing the entire thing in a g (group) with a scale is simplest and keeps the numbers in the paths smaller.
2017-11-28
SVG all the things, now!
It started with "the icons look blurry" from a friend, and I was puzzled, as the icons look fine. We went to a lot of bother to make nice clean PNG images for the icons on the FireBrick config editor.
It turned in to a project taking me at least 4 days so far... I'll never be an artist!
Icons look blurry
Why did they look blurry? Well, the answer is simple - high resolution monitors! I have a 5k Mac, as does my friend, and on such a high resolution screen you just get used to the fact that all pixel graphics look blurry. To be fair, a photograph does not, but an icon, where things should have sharp edges, do not look right. I blogged an example of this recently. Here, I try to go in to a lot more detail.
Here are examples - the first is the original PNG, scaled up showing the square pixels. The second is the blurry version, cubic scaled up as would happen on a monitor that has much higher resolution. The third is a rasterised version of the SVG we now use.
The pixel version was good, it was the best you could do for the pixels available. The scaled version is pretty good, but is indeed blurry. What happens is the sharp transitions on the pixel version are softened. This makes a lot of sense with a photograph as it stops you seeing the harsh edges and makes for smooth transitions in colour.
The SVG though is perfect at any scale, and that is because it is vector based (the V in SVG). It involves lines, and shapes created using points and straight line segments and curves between those points. This is something you could draw at any size. A diagonal line is diagonal and not a jagged series of pixels making a "staircase" effect. Now, ultimately, this ends up on a screen that has pixels, but that conversion to pixels can happen at the last moment for the final resolution of the screen. If you actually had a vector output device, like a pen plotter, it could actually draw the vectors with no pixels.
That is what makes SVG very different. Vector graphics verses raster graphics. The age old issue. But now with computer screens, and even mobile phones, being such high resolution, we need things to be designed from the start as vector graphics. With the possible exception of Minecraft.
Changing this on the FireBrick was simple because we actually have SVG masters for all the icons. Our build process made PNGs from them. On a low level screen all three of those images would look the same in their original size. When all we had were screens that showed images 1px to 1px scale, it looked as perfect as it could.
Sol, given we had SVG, why did we use pixel based PNG. The reason is simple - when FireBrick started browsers did not support SVG, or need SVG as monitors were not so high resolution. Indeed, even javascript was not something on which one could rely. Finally we are at the stage where SVG is pretty much a given, as is javascript. So now we can use those nice vector originals we have had all along.
I can't just leave it alone, can I?
Basically, that was far too easy. It took my minutes as we had the SVG masters of the icons and could just use them.
So this led to my changing the CQM graphs... These are the loss/latency graphs we use for monitoring links, and A&A use for all broadband customers...
Existing graphs were pixel based in the same way, and when viewed on a same resolution screen they look good. They try to present a lot of data, and have options to only show some of the information on the graph, and to adjust scales, and colours and so on. Indeed, the A&A control pages are rather colour-blind friendly by allowing control options for customers to see the graphs in different ways or as multiple graphs even, at least for the "live" graphs.
This is what they look like as a PNG:-
Coverting this to SVG was not so simple - I had to make the raw SVG from the data. I managed to make a lot of common functions to the PNG generation, e.g. text placement and colours, but even that needed work as it made for inefficient SVG. To be honest, it would have been nicer to start from scratch, but I wanted to maintain the layout, at least for now.
A big change was the tx/rx rates which could now sensibly be lines, and not points. SVG makes that very simple.
I did this in all in one day, but have now spent three more days tinkering. It is like a piece of art - I am either fine tuning what you see - moving something half a pixel over, or I am fine tuning the SVG, making it more compact. It is potentially never ending, which is frustrating. I would never make a good artist a I would never finish any piece of work. I even ended up making some generic SVG path optimisation logic to make the paths shorter textually by use of relative points, the "h" and "v" functions, and even not making two segments that could be one segment in a line.
Now, to show you the SVG, but blogspot does not allow me to - WTF?
Fortunately, hosting it and editing the HTML works... So here is the same graph as an SVG:-
This is not the latest version, I will have that on LNSs later this week. I have, for example, moved it 0.5 pixels to the right as the line on the left is one pixel wide at 0 pixels so only shows half a pixel wide! I have fin tuned the size and placement of text (most of which you don't see on this example as it is customer line specific), and optimised a lot of the SVG for things like the "hour" numbers, and so on. This is from at least a day ago... So much has happened since then.
There is one more place...
We present a QR code for Two Factor Authentication TOTP set up on the FireBrick web page. This was generated as a QR code as a simple bit map and then made in to a PNG.
The logic for this already had to make it bigger than it should be, with each QR code pixel being a 4x4 image pixel as simply scaling up in the browser made it "blurry"...
So for the SVG I had to create code to "trace" around the pixels, making an outline for the black parts. Rather than simply making it lots of small squares that butt up against each other I decided to make it actually "trace" the outline of multiple black pixels. It has to trace the "inside" white parts the other way (e.g. counter clockwise), and so on. One thing I was unsure of was whether to follow a "diagonally" joined pixel as part of the same outline - it turned out to save a few bytes as it meant longer paths of "h" and "v" relative points rather than "M" absolutely points for each small part, but it was literally like a dozen characters saved. Even so, the result has no nasty "joins" between adjacent pixels which I have seem break things when scaled up.
The result looks identical to the PNG, e.g. (screenshot):-
SVG is the future of all icons on the web!
The fact we have so many high resolution devices, and scalable screens (e.g. phones) means we need to abandon use of pixel based icons on web pages pretty much. It looks "blurry" normally and just bad when scaled up.
This has turned in to quite a long and fun project, and something we really need to extend to more places, the main web site, control pages and much more.
For now the challenge over the next few weeks is simply switching the A&A control pages to show the new SVG graphs as we update the LNSs. They are already proving very popular. As we have so many LNSs now, this will take a couple of weeks, sorry.
It turned in to a project taking me at least 4 days so far... I'll never be an artist!
Icons look blurry
Why did they look blurry? Well, the answer is simple - high resolution monitors! I have a 5k Mac, as does my friend, and on such a high resolution screen you just get used to the fact that all pixel graphics look blurry. To be fair, a photograph does not, but an icon, where things should have sharp edges, do not look right. I blogged an example of this recently. Here, I try to go in to a lot more detail.
Here are examples - the first is the original PNG, scaled up showing the square pixels. The second is the blurry version, cubic scaled up as would happen on a monitor that has much higher resolution. The third is a rasterised version of the SVG we now use.
The pixel version was good, it was the best you could do for the pixels available. The scaled version is pretty good, but is indeed blurry. What happens is the sharp transitions on the pixel version are softened. This makes a lot of sense with a photograph as it stops you seeing the harsh edges and makes for smooth transitions in colour.
The SVG though is perfect at any scale, and that is because it is vector based (the V in SVG). It involves lines, and shapes created using points and straight line segments and curves between those points. This is something you could draw at any size. A diagonal line is diagonal and not a jagged series of pixels making a "staircase" effect. Now, ultimately, this ends up on a screen that has pixels, but that conversion to pixels can happen at the last moment for the final resolution of the screen. If you actually had a vector output device, like a pen plotter, it could actually draw the vectors with no pixels.
That is what makes SVG very different. Vector graphics verses raster graphics. The age old issue. But now with computer screens, and even mobile phones, being such high resolution, we need things to be designed from the start as vector graphics. With the possible exception of Minecraft.
Changing this on the FireBrick was simple because we actually have SVG masters for all the icons. Our build process made PNGs from them. On a low level screen all three of those images would look the same in their original size. When all we had were screens that showed images 1px to 1px scale, it looked as perfect as it could.
Sol, given we had SVG, why did we use pixel based PNG. The reason is simple - when FireBrick started browsers did not support SVG, or need SVG as monitors were not so high resolution. Indeed, even javascript was not something on which one could rely. Finally we are at the stage where SVG is pretty much a given, as is javascript. So now we can use those nice vector originals we have had all along.
I can't just leave it alone, can I?
Basically, that was far too easy. It took my minutes as we had the SVG masters of the icons and could just use them.
So this led to my changing the CQM graphs... These are the loss/latency graphs we use for monitoring links, and A&A use for all broadband customers...
Existing graphs were pixel based in the same way, and when viewed on a same resolution screen they look good. They try to present a lot of data, and have options to only show some of the information on the graph, and to adjust scales, and colours and so on. Indeed, the A&A control pages are rather colour-blind friendly by allowing control options for customers to see the graphs in different ways or as multiple graphs even, at least for the "live" graphs.
This is what they look like as a PNG:-
Coverting this to SVG was not so simple - I had to make the raw SVG from the data. I managed to make a lot of common functions to the PNG generation, e.g. text placement and colours, but even that needed work as it made for inefficient SVG. To be honest, it would have been nicer to start from scratch, but I wanted to maintain the layout, at least for now.
A big change was the tx/rx rates which could now sensibly be lines, and not points. SVG makes that very simple.
I did this in all in one day, but have now spent three more days tinkering. It is like a piece of art - I am either fine tuning what you see - moving something half a pixel over, or I am fine tuning the SVG, making it more compact. It is potentially never ending, which is frustrating. I would never make a good artist a I would never finish any piece of work. I even ended up making some generic SVG path optimisation logic to make the paths shorter textually by use of relative points, the "h" and "v" functions, and even not making two segments that could be one segment in a line.
Now, to show you the SVG, but blogspot does not allow me to - WTF?
Fortunately, hosting it and editing the HTML works... So here is the same graph as an SVG:-
This is not the latest version, I will have that on LNSs later this week. I have, for example, moved it 0.5 pixels to the right as the line on the left is one pixel wide at 0 pixels so only shows half a pixel wide! I have fin tuned the size and placement of text (most of which you don't see on this example as it is customer line specific), and optimised a lot of the SVG for things like the "hour" numbers, and so on. This is from at least a day ago... So much has happened since then.
There is one more place...
We present a QR code for Two Factor Authentication TOTP set up on the FireBrick web page. This was generated as a QR code as a simple bit map and then made in to a PNG.
The logic for this already had to make it bigger than it should be, with each QR code pixel being a 4x4 image pixel as simply scaling up in the browser made it "blurry"...
So for the SVG I had to create code to "trace" around the pixels, making an outline for the black parts. Rather than simply making it lots of small squares that butt up against each other I decided to make it actually "trace" the outline of multiple black pixels. It has to trace the "inside" white parts the other way (e.g. counter clockwise), and so on. One thing I was unsure of was whether to follow a "diagonally" joined pixel as part of the same outline - it turned out to save a few bytes as it meant longer paths of "h" and "v" relative points rather than "M" absolutely points for each small part, but it was literally like a dozen characters saved. Even so, the result has no nasty "joins" between adjacent pixels which I have seem break things when scaled up.
The result looks identical to the PNG, e.g. (screenshot):-
SVG is the future of all icons on the web!
The fact we have so many high resolution devices, and scalable screens (e.g. phones) means we need to abandon use of pixel based icons on web pages pretty much. It looks "blurry" normally and just bad when scaled up.
This has turned in to quite a long and fun project, and something we really need to extend to more places, the main web site, control pages and much more.
For now the challenge over the next few weeks is simply switching the A&A control pages to show the new SVG graphs as we update the LNSs. They are already proving very popular. As we have so many LNSs now, this will take a couple of weeks, sorry.
Today's recipe: Ryvita and soft cheese
Well, I am pretty sure that Ryvita is probably reasonably healthy...
The problem is what you do with the Ryvita...
I did not really get a body like this by eating healthily, did I?
So here's the recipe (if you can call it that)...
The problem is what you do with the Ryvita...
I did not really get a body like this by eating healthily, did I?
So here's the recipe (if you can call it that)...
- Starts off well...
- But then loads of butter...
- Ryvita and butter is all very well, but actually, it is quite nice to toast it, so the butter melts in to the Ryvita...
- You want it to all soak in nicely...
- Then you'll want the marmite, obviously...
- On the Ryvita...
- And maybe some soft cheese, this is quite nice...
- Delicious :-)
Bloody banks!
As I have previously blogged, I made the mistake of guaranteeing an Amigo loan for someone. My personal opinion is never ever go near Amigo loans, and don't even fall for guaranteeing a loan for someone else.
In the end, we asked Amigo for a settlement figure, and I paid it. My friend is now repaying me. Yes, I know, "I told you so" all round.
Amigo loans
I asked for a "settlement figure" and received one, after a long and very shouty call as I mentioned in a previous blog (sadly not recorded). I got the settlement figure by email dated 19:30 on 2nd October. I logged in to my Barclays on-line banking and paid it in fill. Amigo have now confirmed in writing that the payment was "accepted" as part of the "faster payments" process at 19:37 on 2nd. Yay!
You would think that is the end of the matter, but no.
Firstly, it is clear that Amigo are unable to provide a "settlement figure", as even paying them in full in irrevocable cleared funds in only 7 minutes of getting the figure, it was not, according to them, sufficient to "settle" the account. There was another £10 of interest to pay. I am pretty sure that the whole point of a "settlement figure" (which is what they called it in the email) rather than just an "account balance" is that it is a figure that can "settle" the account. The fact they are unable to actually provide one is a serious issue in my view, and so has been reported to the Financial Conduct Authority.
So why was there interest still to pay? Well, it seems, according to their written reply to me, that they deliberately delay processing payments until the following working day. Well, I paid on the 2nd, but they recording the payment as being paid on the 4th. It seems that "following working day" was a bit of a lie, as it can be the day after that.
Their reason for this? That "payments can be subject to confirmation or be reversed"!
Well, that is true for some types of payment, but waiting one day does not help. Direct Debits can be clawed back at any time, so clearly they have a process for handling payments being reversed - how does delaying things help. However, Faster Payments cannot be reversed (as per FAQ on the faster payments web site), so not actually a valid reason anyway.
My issue though, regardless of how much they delay processing the payment they have received, is that they should record the payment on the account correctly - the correct amount, and surely, the correct date. The date they were paid. I have no issue with them waiting until the 4th to record the payment, but record it as having been paid on the day it was paid, i.e. the 2nd, and so apply any interest on that basis.
To deliberately falsify the record of when a payment was received by at least one working day on every payment, in order to collect more interest, seems to me to be a simple matter of criminal Fraud.
So, reported to Financial Conduct Authority for that as well.
Barclays
Barclays worked well in this case, the faster payment did work, and arrived in seconds. Barclays have a "tracker" page on on-line banking that you can use to check such a payment has been made. They can confirm that it has been "accepted" by the receiving bank. I did this at the time, and saw it was accepted.
However, it was not until nearly two months later that Amigo finally actually sent a statement saying the payment was made at all, and that it was made on the 4th, not the 2nd, and that interest was due. I was fuming, obviously, and wanted evidence that I had paid. I checked Barclays on-line banking and the "tracker" only goes back one month, so I could not get the evidence, damn. All I had was a statement showing the banking day (3rd).
So, simple, there is a secure messing system. I simply asked Barclays if they can confirm when the payment was received. Not a hard question, surely.
Their first reply, after a few days, was that they cannot confirm when it was actually received, but I could use the tracker to check! Well, I explained in the question that the tracker did not go back that far, so they clearly had not in fact read the question at all!
They also suggested I simply ask the recipient, so I pointed out that we had a dispute about when it arrived, which is why I am asking Barclays! My original email did not explain that, so fair enough.
I replied, explaining again that the tracker did not go back that far, and asking if instead they could confirm when Barclays received the acknowledgement of the payment having been made. Maybe the phrasing of my first question had been unclear.
Finally they come back saying that my account was debited on the 3rd (duh, I can see that on the statement, and not what I asked), and that the recipient would have had it "within or after 24 hours".
Let's think about that...
In the end, we asked Amigo for a settlement figure, and I paid it. My friend is now repaying me. Yes, I know, "I told you so" all round.
Amigo loans
I asked for a "settlement figure" and received one, after a long and very shouty call as I mentioned in a previous blog (sadly not recorded). I got the settlement figure by email dated 19:30 on 2nd October. I logged in to my Barclays on-line banking and paid it in fill. Amigo have now confirmed in writing that the payment was "accepted" as part of the "faster payments" process at 19:37 on 2nd. Yay!
You would think that is the end of the matter, but no.
Firstly, it is clear that Amigo are unable to provide a "settlement figure", as even paying them in full in irrevocable cleared funds in only 7 minutes of getting the figure, it was not, according to them, sufficient to "settle" the account. There was another £10 of interest to pay. I am pretty sure that the whole point of a "settlement figure" (which is what they called it in the email) rather than just an "account balance" is that it is a figure that can "settle" the account. The fact they are unable to actually provide one is a serious issue in my view, and so has been reported to the Financial Conduct Authority.
So why was there interest still to pay? Well, it seems, according to their written reply to me, that they deliberately delay processing payments until the following working day. Well, I paid on the 2nd, but they recording the payment as being paid on the 4th. It seems that "following working day" was a bit of a lie, as it can be the day after that.
Their reason for this? That "payments can be subject to confirmation or be reversed"!
Well, that is true for some types of payment, but waiting one day does not help. Direct Debits can be clawed back at any time, so clearly they have a process for handling payments being reversed - how does delaying things help. However, Faster Payments cannot be reversed (as per FAQ on the faster payments web site), so not actually a valid reason anyway.
My issue though, regardless of how much they delay processing the payment they have received, is that they should record the payment on the account correctly - the correct amount, and surely, the correct date. The date they were paid. I have no issue with them waiting until the 4th to record the payment, but record it as having been paid on the day it was paid, i.e. the 2nd, and so apply any interest on that basis.
To deliberately falsify the record of when a payment was received by at least one working day on every payment, in order to collect more interest, seems to me to be a simple matter of criminal Fraud.
So, reported to Financial Conduct Authority for that as well.
Barclays
Barclays worked well in this case, the faster payment did work, and arrived in seconds. Barclays have a "tracker" page on on-line banking that you can use to check such a payment has been made. They can confirm that it has been "accepted" by the receiving bank. I did this at the time, and saw it was accepted.
However, it was not until nearly two months later that Amigo finally actually sent a statement saying the payment was made at all, and that it was made on the 4th, not the 2nd, and that interest was due. I was fuming, obviously, and wanted evidence that I had paid. I checked Barclays on-line banking and the "tracker" only goes back one month, so I could not get the evidence, damn. All I had was a statement showing the banking day (3rd).
So, simple, there is a secure messing system. I simply asked Barclays if they can confirm when the payment was received. Not a hard question, surely.
Their first reply, after a few days, was that they cannot confirm when it was actually received, but I could use the tracker to check! Well, I explained in the question that the tracker did not go back that far, so they clearly had not in fact read the question at all!
They also suggested I simply ask the recipient, so I pointed out that we had a dispute about when it arrived, which is why I am asking Barclays! My original email did not explain that, so fair enough.
I replied, explaining again that the tracker did not go back that far, and asking if instead they could confirm when Barclays received the acknowledgement of the payment having been made. Maybe the phrasing of my first question had been unclear.
Finally they come back saying that my account was debited on the 3rd (duh, I can see that on the statement, and not what I asked), and that the recipient would have had it "within or after 24 hours".
Let's think about that...
- Faster Payments have a process that includes an acknowledgement. Barclays know they have that as they show in their tracker. So either Barclays fail to actually keep records, which would be a surprise, or they are just incompetent and cannot look them up.
- Faster Payments are normally within seconds, and otherwise up to 2 hours. Why suggest 24 hours?
- What does "within or after 24 hours" actually mean. It seems to cover the time period from when the payment was sent up to 24 hours later, plus the entire time from 24 hours later to the to the final heat death of the universe. So basically "some point after you sent the payment". The "24 hours" part of the sentence actually adds no meaning whatsoever. You have to wonder if they get training in being vague and meaningless in their replies.
So, now I have a complaint in to Barclays over their handling of a simple enquiry regarding a payment.
Why is this so damn hard?
No, A&A are not going to start a bank, sorry.
2017-11-23
SVG
I mentioned websockets recently as an awesome but little known feature of most browsers - the other is SVG (Scalable Vector Graphics) as an image format.
For so many things a vector based image / icon is way better than a pixel based one. Yes, jpeg for actual pictures of stuff, but icons and so on want to be drawn not imaged.
So a simple example, FireBrick config pages, as they have always been, using road signs... I am used to them. But on my 5k mac I did not realise they are maybe "blurry"...
Change to SVG and now they are not...
Maybe I need to blow that up a bit - screens shots on my Mac... Still these are bitmap screen shots.
For so many things a vector based image / icon is way better than a pixel based one. Yes, jpeg for actual pictures of stuff, but icons and so on want to be drawn not imaged.
So a simple example, FireBrick config pages, as they have always been, using road signs... I am used to them. But on my 5k mac I did not realise they are maybe "blurry"...
Change to SVG and now they are not...
Maybe I need to blow that up a bit - screens shots on my Mac... Still these are bitmap screen shots.
Impressively different, but the latter can be blown up indefinitely as it is vector graphics...
So really all icons need to be SVG, it is cool, and it just works.
P.S. here is what happens when CQM graphs are done as SVG!
P.P.S. I have actually spent two whole days on this, annoyingly - largely because it is a bugger to test, and even testing locally you then want to test on hundreds of different graphs for several hours, any which point you find something is like half a pixel out when you zoom in - arg, bring back pixels, all is forgiven. Anyway, several firebrick alpha releases today. Looking damn good, and yes, we will be rolling this awesomeness out to A&A customers over the coming weeks.
Subscribe to:
Posts (Atom)
Fencing
Bit of fun... We usually put up some Christmas lights on the house - some fairy lights on the metal fencing at the front, but a pain as mean...
-
Broadband services are a wonderful innovation of our time, using multiple frequency bands (hence the name) to carry signals over wires (us...
-
For many years I used a small stand-alone air-conditioning unit in my study (the box room in the house) and I even had a hole in the wall fo...
-
It seems there is something of a standard test string for anti virus ( wikipedia has more on this). The idea is that systems that look fo...