Notes from BBC Click March 23rd 2019

Click looks at the latest anti-drone tech from the Netherlands, Google’s move into online gaming, and making coding accessible for children with vision impairment. 

Controlling unlawful use of drones.

Last years disruption at Gatwick was reported to have cost £50m. So how do you take down problem drones safely?

DroneClash 2019 Champions

A compétition set in the Netherlands sponsored by the police. Click interviews Mark Webes Invitations Manager at the Dutch National Police. He was hoping to see new strategies for downing drones but he also saw new threats. Drones encased in balls are less vulnerable to nets and other tactics for interfering with their rotor blades.

Bart Remes on of the Co-organisers of DroneClash 2019 shows off one of the microdrones. He invites the public to try and hack them while they are flying around the event.

Three years ago the Dutch police introduced trained eagles to down drones. The programme ended a year later as the birds could not be called on fast enough or down them cleanly enough.

Bart Van Maarle Captain Team Speeddrones is shown fixing drones which have been downed or damaged in the contest.

Stefaan Joos, SEO JBT Security, in a normal environment when you jam the frequencies used by a drone it will start hovering, or go back to its operator or drop when the battery fails.

Sjors Van OS, Development Engineer at Delft Dynamics, talks about how their radar system is used to line up the drone and then fires a net over the drone and then causes it to drop on a parachute.

This gun fires a powerful EMF (Electro Magnetic Field) pulse at the drone overpowering its control and command systems. It’s illegal in Europe, probably because logically if it works on a drone it could also seriously damage anything else that’s based on microelectronics such as cars and planes. The makers claim it’s selling well in other parts of the world.

Andrew Scheer, Marking Director at Metis Aerospace talks about the Electromagnetic Spectrum Tracking technology drone detection systems now being deployed at British airports. By having two base stations they are able to triangulate on the position of a drone.

Notes from the week in tech

Stage Coach were shown trailing a full size driverless bus.

Google unveiled their new gaming platform Stadia at the Game Developers Conference. (GDC)

Google being fined £1.82bn by the EU for blocking rival search engines.

MySpace announced they had lost 12 years worth of audio and video files during a server migration project.

GoGap from NVidea was shown transforming rough sketches into natural scenes in realtime.

Google Stadia

Phil Harrison, Head of Stadia at Google talks about how they have designed the new games platform so that it doesn’t have any lag. They can deliver up to 4K, at 60 frames per second, with HDR and surround sound to players devices, depending on their available bandwidth. He says that with the advances they are making with compression algorithms they can increase the quality without necassarily needing to increase bandwidth.

Google hasn’t explained the business model yet but most people expect it to operate like a Netflix for video games. They are expected to announce the launch date and pricing at the E3 Games Conference in June.

Coding for Blind and visually impaired children

This is Code Jumper a physical, tactile coding language developed by Microsoft. It’s a block-based module system consisting of a series of pods that contain a single line of code.

This was developed by Cecily Morrison who works as a researcher at Microsoft Research in Cambridge.

Children typically start learning programming these days at age 7 with block based languages like Scratch. This system is good for all children at this stage in their development not just those with visual impairment.

Apple are also working to extend access to visually impaired people to their Swift Playgrounds development environment.

Robin Spinks, Principal Manager Digtal Accessibility at RNIB, talks about how brail instructions and the accessibility features of the iOS are used together.

Medical Scanner Feature

The BodyO pod is designed to give you a full body medical in ten minutes without seeing a doctor or needing to make an appointment. Click spoke to Dave Boston the Business Development Director at BodyO. It’s a great way to encourage people to watch their weight and fight diabetes.

What is CI/CD?

As a combined acronym I think CI/CD entered common usage around 2018. The CI part stands for Continuous integration (CI) and the CD part stands for continuous delivery (CD). Both of which have been around for a long time.

To understand CI/CD lets break it down:

So what is Continuous Continuous (CI)

Let’s try and explain CI for non-technical / non-development staff. A software product can be thought of in a similar way to a large complex storybook. The book might be written by one author over a number of months of years but if there’s commercial pressure on getting the book published it may be written by a team of authors who collaborate on the work.

In the case of a book, the work may be shared out between the authors, illustrators and editors. These might be split into separate teams, with one team responsible for the architecture of the story (ie the plot, characters, setting etc) and other teams assigned to various chapters.

The publishing house could wait until each team says they have finished work and then ask each team to send in their chapters and illustrations and then give some poor editor the job of trying to merge it all back together again.

The merge process would likely take days or weeks of pasting text together which wouldn’t be very efficient or these days commercially viable. However worse than this, the likelihood is that when pasted together the chapters no longer make a coherent storyline. For example, a key character needed in chapter 10 may never have been overlooked and not introduced in the earlier chapters or might have been dramatically murdered with pages and pages of text devoted to this scene.

If this sounds like a nightmare way of writing a book then you’ll be horrified to learn that this is precisely the way programmers/developers work together on large software development projects. The more developers that are involved the greater the risk of misunderstanding and error. Teams of more than 6 or 7 developers actually get less efficient but the management response to a project which is running late or which needs to deliver by a fixed date is usually to throw more people at it. (Developers joke about inexperienced project managers asking 9 women to have a baby in a month. )

In the case of a book written by teams of authors, the publisher could try and avoid these problems by having each author submit their latest text each day or even work in an online shared document. The editors can then see how the work is going and immediately jump on any mistakes while it’s easy to put them right. In fact, it may not even need the editors to spot the mistake, the chances are the authors, being able to see each other’s work will read the chapters others have written and spot continuity errors for themselves.

I’m sorry to say that software is more complex than writing a book. A continuity error in a book isn’t going to cause the financial failure of a company or kill someone but software does break companies and can kill when it goes wrong.

In software development, we, therefore, ask developers to ‘check in’ the code they are working on daily into a central repository which all the other developers then sync with. The master code in the repository is then compiled (turned into machine executable code) in a process known as the ‘build’.

To spot structural errors and continuity errors in the code the build process can include static analysis of the source code to spot frequently recurring errors and breaches of house style and automated tests.

Automated tests are scripts which run parts of the codebase with known input and test to see if the code generates the expected outcome. For example, in an accounting system, a test might post a debit and a credit and then test the account balance is correct. The same test might be run with different scenarios for example what happens if the debit takes the account overdrawn or what happens if a very large amount is deposited. In any case if one of these automated tests fails then the build fails and the team focus on fixing the error before continuing with any other development work.

So what is Continuous Deployment (CD)?

Going back to our book analogy for a little while, lets say the publisher merges together all the text, and maybe automates a spell check and grammar check over the document. If these don’t find any errors then they might choose to automatically deploy a draft version of the new books in eBook form to readers who they know are interested in the forthcoming title and who would be willing to provide constructive feedback. This then is continuous deployment of the book on a daily basis.

The publisher could employee someone to manually send the draft of the eBook or upload it to a website each day, but doing this manually would be inefficient and error prone. If the publisher decides to move from daily updates of the ebook to hourly updates, around the clock doing the work manually become even less viable. 

The publisher will look for ways to automate the deployment process of the eBook initially to the proofreaders and ultimately to the eBook stores on Amazon and Apple. This means the task will be performed consistently and can be performed frequently very efficiently. 

The same thing happens with software. Instead of letting a group of readers have access to the book, the software is deployed into a user testing environment. Depending on the size and scale of the project this might be a UAT (User Acceptance Test) environment or a Staging Environment or indeed another entirely separate testing environment for the project. 

The process for deploying software into a test or production environment can be very complex, one mistake can be the difference between success and failure. It’s not something you want to be doing manually especially when you consider that many leading organisations will be deploying software updates hundreds of times as day. Thankfully there are tools which allow the whole process to be scripted. This can cover everything from defining the virtual environment that the application runs on through to the software itself.

CI/CD Summary

In business today success is often governed by how quickly the organisation can produce the next iteration of their product, it’s so important it has its own technical term called cadence. Let’s be clear, if two organisations are identical other than for their rate of cadence, the one with the fastest cadence will win.  This can be tracked back to the laws of evolution and the ability to react and respond the (commercial) environment.  

This has been a non-technical explication of CI/CD for executives, directors and managers. If you’d like to learn more about this there are a number of good books on the subject.  The best place to start is Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation.

How to watch The Orville in the UK

How to watch The Orville in the UK

Orville,Star,Trek

 

I know this is a tech blog, but many of us grew up watching Star Trek and that may in part be responsible for why we now work in tech.  While I’m sure we’re all looking forward to the new Star Trek Discovery we’re also excited about The Orville. The first episode of The Orville will air on September 10th in the US on Fox.

As of September 3rd, no details have been given of how or when the Orville will be released in the UK. This is a little strange these days when we are used to global distribution arrangements.

We are monitoring this on a daily basis. We expect it to at the very least become available on Amazon, Netflix or Apple iTunes stores around the time it launches in the US.  The episodes may also be posted onto YouTube as the studio would be sure to make a fortune off the advertising income.

How to improve MySQL 5.7 Performance

How to improve MySQL 5.7 Performance

MySQL 5.7 How to fix performance issuesWe upgraded to MySQL 5.7 and have had numerous problems with performance. We upgraded because we needed the new support for fractional timestamps but we’ve since learnt that this functionality has been ported back to MySQL v5.6.35.  So in some situations the best way to fix MySQL 5.7 performance issues is to downgrade to V5.6.35 which actually came out after 5.7.

We also struggled with some indexes which MySQL 5.7 ignored despite everything we tried. The queries kept coming back up in the slow running query log. These indexes then just started working after we had rebooted the whole server for another reason. So if you are finding MySQL 5.7 slow, it’s worth trying a reboot.

The other problem we had was that 5.7 can take a very long time to startup. We still haven’t found a solution for this other than to downgrade back to 5.6.35. It in part seems to be caused by having very large record sizes on some tables but then we also saw it affecting some very short tables. It seems to be that after a restart MySQL 5.7 rebuilds the query optimisation statistics cache and buffer pools each time. In previous versions these can be preserved when the Database is closed down. We haven’t managed to make this work in 5.7.

In my view as of January 2017 5.7 is not production ready and the only real solution to addressing the performance issues is to downgrade back to v5.6.35.  For the record I’ve found MySQL V5.6.35 reliable and performant so far.

AS400 Operating System Introduction

AS400 Operating System Introduction

So what makes the AS/400 Operating System so special?

The AS400 Operating System is one of the most modern Operating Systems in wide spread use today.  I was at an interesting talk given by Doctor Frank Soltis the chief designer for the IBM AS/400 and the IBM System 38 that came before it. He explained that Windows can trace it’s roots back via Windows NT to VAX VMS, Unix, Linux and OSX can trace their roots back into the late 60’s. In comparison OS/400 or CPF (Control Program Function) as it was known on the System/38 was designed the ground up in the 1980’s.

In current terms OS/400 has more in common with the Java Virtual Machine (JVM) than operating systems like Windows. OS/400 has a machine independent layer that sits between it and the underlying processors and hardware.  Over the years the processors have changed many times, for example from CISC to RISC whilst marinating software compatibility. Software first compiled in the 1980’s on long forgotten hardware can still be run on the latest hardware today without being recompiled.

In addition to the Technology Independence Layer, OS/400 also features something called Single Level Storage.  In computer software the most expensive operation is the Branch operation. This is where the processor switches from one task to another.  OS/400 handles this by having a single continuous 128 bit  address range. This covers all the physical memory and disk storage. The Operating System then uses heretical storage management to move the most frequently used content into the fastest memory in the Single Level Storage. These days this will range from slow mass storage drives, SANS, fast drives, SSD’s to RAM.

OS/400 is not a truly Object Orientated Operating System in the current sense (i.e. inheritance and polymorphism) but it is Object Based and very much embraces Encapsulation. Being Object Based literally everything on the system is an Object. Each object has a set of interfaces and can only be accessed through those interfaces. This has made the platform incredible secure as most of the memory overrun and buffer attacks that hackers exploit on other platforms are not possible.

The other interesting thing about OS/400 is that it has a built in database. These days it’s called DB/2 and shares much of the DB/2 technology from other platforms. OS/400 uses this built in database for many of it’s own storage requirements. DB/2 on iSeries leverages the Single Level storage to be highly reliable and low maintenance.  Unlike on other platforms the AS/400 will run for years without any operator or DBA (Database Administrator) assistance.  For example I’m aware of several companies in the UK where AS/400 applications run for years on end with little to no operations support. I’d contrast this with Microsoft SQL Server based applications where performance degrades if not given regular attention.  IBM S/38 was the first commercial computer to implement a Relational Database using SQL. This took the conceptual work from Ted Codd at IBM and implemented it for real in the S/38.  At the time the S/38 CPF Operating System was ahead of the processing power really available for this type of processing. It was slow but successfully proved the vision of a Relational Database based on SQL. IBM had invested everything in the development of the S/38 and continued this investment as they improved the processors, hardware and Operating System and relaunched it in the 1990’s as AS/400.

The AS/400 as a whole is often perceived to be out of date (by people who don’t know better) because it still runs many Green Screen or Character based applications on dumb terminals or terminal emulation.  From a systems architecture perspective this is no more relevant than say the console interface on Mac OS (OSX) or the Terminal Interface on Windows. These days most modern applications are browser based and the users would never know that the backend is running on OS/400.

For anyone studying Systems Architecture or Computer Science OS/400 is something you really must study to get an alternative view of how well a fully integrated, single vendor system can work on an enterprise scale. Few doubt this today on the desktop or iPhone when they look at Apple products.

Do you have any IT Questions?

Any IT Questions?

I’m looking for some good IT Questions covering Windows, Mac, WordPress, iPhone, Website Development, project management, telecoms etc that I can research and write up as interesting posts on this blog.  If you’ve got any good questions that you’ve not really been able to find the answers to elsewhere online please leave the question in a comment below.  I can’t promise to answer it but I’ll have a good go at it.