Mastering Embedded Systems

I have recorded this episode twice. Not by intention – far away from that. But I was neither convinced nor satisfied with the first recording. Also the subject was not really impressive and I decided to do it again from scratch. Thus I got a first hand impression what waste of time and effort is. However one detail in the first record was amazing. It was about waste in SW-development.

You remember episode 4? Already there I have mentioned waste due to over-processing and over-engineering. Both are part of the 7 forms of regular waste in any kind of system.

The Toyota Production System (TPS) has introduced to a broader audience the classification of waste forms. In this episode I wanted to connect traditional manufacturing based Kaizen and Muda, Mura, and Muri with Software-development. Is it possible? Do we have parallels? Or is the TPS not adaptable to nowadays IT-based technology?

Stay with me and enjoy the show.

Kaizen is Japanese and means Improvement. It is regularly used in conjunction with Continuous Improvement. However that’s not the original intention. Kaizen stands for the understanding and the wish that everything can be improved.

The classification of wastes for manufacturing and assembly purpose is one of the key-concepts in the Toyota Production System (TPS) as one of the three types of deviation from optimal allocation of resources (muda, mura, muri).[2]. Waste reduction is an effective way to increase profitability.

Lean and TPS knows the key step to distinguish between manufacturing steps which add value and which don’t. The classification of all processes involved into these two categories let one improve the one and eliminate the other. That means Muda is any activity or process which does not add value. Thus it is a physical waste of time, effort, resources and finally money.

In principle can all forms of waste in Muda be used for SW-development, too. However some of them are quite seldom and do not effectively match always, like Transport or Motion. However others, like Over-Production, Over-Processing and of course Defects seem like exclusively made for SW-development.

Mura and Muri define some more abstract ways how waste can be finally created in Muda-categories. Mura and Muri drive Muda.

Mura stands for the waste of unevenness. If the available men, material and equipment are used in a non-balanced way, this finally will result in waste. For example, four weeks ahead of an upcoming milestone the previous measured way of working is changed into a more hectic, confuse and challenging action. The probability of waste, especially defects, will increase. This can be prevented by steering the utilization of all resources in a balanced way.

Muri stands for the waste of overburden. If there is unnecessary stress given on employees and processes this will cause Mura and then Muda. Permanent overburden will cause absences of employees, increase of sickness-rate and finally a higher attrition rate.

[saf feature="email" cta="Send me feedback"][saf feature="twitter" cta="Reply on Twitter"]

You remember episode 20 where I mentioned the Embedded Testing conference in Munich. One of the presentation was about Applying DevOps Principles to Software-Hardware Integration Test. Vassilis Rizopoulus explained details about DevOps, their tooling, automated testing and the mandatory environment.

I was excited and we have had an extraordinary talk after his contribution. Some days later I asked him for an interview. I am sure that Vassilis’ ideas and thoughts will be helpful for a lot of engineers out there. Especially engineers who wanted to try something knew, but who might need some trigger to change their way of thinking and habit.

Vassilis has been dabbling in software engineering close to two decades by now. In this time what started as a hobby of a would-be electrical engineer became full time occupation. He specializes in software productivity engineering, a catch all term for the role that integrates the development environment, test automation, continuous integration and deployment, devops and general behind the scenes tooling that enables software teams to concentrate on actually producing useful software.

His language of choice is Ruby and has been for the past dozen of years, but he also has several C and C++ embedded projects under his belt as well as a few C# .NET projects. Most of his professional career has been spent working for large industry firms doing really close to the metal stuff, from devices smaller than an Oreo cookie to as large as 60-ton locomotives. He is one of the co-founders and organizers of thessaloniki.rb the Thessaloniki Ruby Users group and also had the luck to be part of the organizing committee for EuRuKo 2013 that took place in Athens. Whatever free time remains between deadlines and family life goes into open source projects mainly in the Ruby community with the lion’s share reserved for rutema and gaudi.

In the Engineers’ Talk we’re discussing about DevOp-priniciples. His will to automatize everything. The infrastructure you need for day-by-day development. And of course the mother of all questions: what are the differences between SW-tester and SW-developer.

Stay with me and enjoy the interview.

In episode 20 I have described the main differences between traditional conferences and the idea of BarCamps. In this session I wanted to give you my 15 tips that your first BarCamp visit becomes a success.

I have meanwhile joined several BarCamps. And I have made a lot of things wrong. Mainly because I was not aware of. Sometimes also I thought it is better done in another way. But there are a some main phases you are confronted with. And for all of them you can do things well and you do things much more better.

Starting into a BarCamp well prepared lets you enjoy the atmosphere even more intensively. You get much more quicker into connection with the other BarCamp-members. And you finally get more out of the time you’re staying at the event.

All BarCamps can be separated in three different phases: BEFORE, DURING and AFTER the BarCamp. Even the AFTER phase is as much important as the other phases. If you do not care for the AFTER-phase you loose a lot of momentum you have gained during your time before.

Please take these 15 tips into account. Let your first or next BarCamp become an even more thrilling success.

BEFORE the BarCamp it is much important to prepare yourself than in traditional conferences. In traditional conferences you more or less get surprised by the presenters and the program. On BarCamps the program will be designed and composed by all contributers. Therefore it is essential to get the idea and perhaps already the spirit of the BarCamp you’re willing to join. Even more, if you want to get the best out of it, a well preparedness is essential. Let’s start into it:

BEFORE the BarCamp it’s all about communication. Prepare yourself for communicating a lot. Not only verbally, but mainly also by Internet. The main intention is to get familiar with upcoming participants.

DURING the BarCamp could be considered the most important phase. However in my experience it is not. It might be of course the longest phase, but without your good preparation in the BEFORE-phase you might not get the full potential.

DURING the BarCamp it’s all about getting the most out of it. In what direction? That depends on you. But you should have an agenda, an intention, which you have prepared in your BEFORE-phase.

AFTER the BarCamp is different from regular traditional conferences. As your connection on the BarCamp was on eye-level and you’re already connected with other participants via Twitter, the BarCamp still continues for a while. It slowly goes down in the next two or three days. But everybody has the common days in mind and is eager to recapitulate or remember things.

In the AFTER the BarCamp phase it’s all about doing the aftermath, providing details, hardening your contacts and stay in touch with the participants.

Holy moly, seems I’m getting old. Two times in the last two weeks I was confronted with this fact. Things which are part of my experience are nostalgic for others. They have never seen or touched them.

What has happened? One day I saw an update in LinkedIn highlighting DEC’s PDP-11 as an old system. That’s nothing extraordinary. It is old. But one of my colleagues mentioned it as nostalgic. Bumm, right in the face.

Second, I got a mail by another colleague. In his signature he mentioned that this mail was written from his PDP-11. I doubted this, but he showed me a picture showing him from the backside (hopefully it is him) and a PDP-11 aside. That picture must be old. I do not consider that he really has a PDP-11 in his cellar.

But both moments let me think back into that time 30, 40 years ago. Computer Systems were different. Computer Science was slightly different. And of course the systems you own or you could directly touch were dramitically different. Compared with the systems available nowadays the are more like flintstones than real lighters.

And these thoughts created this episode. I wanted to go back with you showing some highlights in my computer-life. Not for nostalgic reasons, but to make you aware that things were really different at that time. And that it is of some benefit to know a little bit about it, to understand the situation we’re currently in.

Please pay special attention on this video showing the startup phase of the PDP-11.

Together with its printer and its magnetic card it was – more or less – a fully equiped computer system. The price was horrible. But it still was only a fraction of a real computer system.

As each punching card is representing one line of your batch-job, you should take care that the order is not corrupted. Stumbling or shuffling has to be prevented at any price.

Mounting of devices at that time really meant that someone, the operator, physically lifted the storage into a storage-equipment. If you liked to mount some storage you do not have privileges for, the operator might come personally to your working place offending you as stupid moron.

The Atari ST was the first fully graphical desktop system. With its GEM Operating System it opened a whole new world. Out of the familiar character based limitation into the freedom of fully flexible presentation.

The PDP-11 has had to be started by using the front-panel in-line keys. These keys were set representing the address and the octal data to setup the boot-code.

If you like to deal with computers on a more technical level you should know their very basic language. And assembler is mnemonized object-code. If you know assembler you are directly in touch with the bare metal. Mandatory if you need to know for sure what’s going on.

You know conferences. For sure you do. You might be a fan of them. Or not. You might be disappointed. As I was after visiting the Embedded Testing 2015 in Munich last week. Although I was invented as presenter, I was not convinced. I mean the topic was really good – no other conference or meeting available covering exactly this specific part in the Embedded Systems realm.

But the conference got stuck as a sponsor-driven event which highlights products over general knowledge, marketing over sharing of experience. One could have made a lot more out of this conference. So the participants have finally had only their others to get acquainted with each other, hear the problems, discuss and finally detect the old rule, that the best of conferences are the pauses. Free time which can be used for connecting and sharing.

In this episode I wanted to highlight the approach of traditional conferences in comparison to BarCamps. A BarCamp is an un-conference born from the desire for people to share and learn in an open environment. It is the direct response on all attendees who wanted to create and design their own agenda of content.

For me BarCamps, the un-conferences are by far the better conferences. Listen to this episode and create understanding about this popular approach to confer.

Attendees regularly expect sharing of information, knowledge experience. They very often expect ideas and impulses for their own problems. They regularly focus on problem solving. Instead they get sponsors’ driven events only highlighting their own products and particular approaches.

The opportunity to make an impact on the sessions, their schedule or the selection of presentations. Very often there are only one or two presentations you’re really interested in.

As many of the conferences are sponsored the companies behind try to monetize their presentation for their own purposes. That often results in pure product presentation and selling event. This again thwarts the expectations of the audience.

BarCamps are un-conferences not following the traditional approach of given schedule and predefined structure of speakers. People share and learn in an open environment not preferring the presenter over the audience.

There are several pre-defined as also inofficial rulse. Listen into the episode to get some of them in detail. The general rules could be also find at BarCamp-rules.

If you want to experience know-how and get acquainted with interesting characters out of your environment or industry. You should join BarCamps if you’re interested more in sharing of expertise and problem-solving experience than in particular products.

There are no spectators, but only participants. It is expected that you do not only consume, but actively engage yourself into the sessions, the discussion and share your knowledge.

Talking about bug-hunting has become a regular topic in this podcast. However there are tons of different persons out there using their own approach to handle bugs. All of them might have different attitudes, different approaches and different experiences. And it might be of interest to get more familiar with them and the details they can tell us.

Within today’s episode we have Bero Brekalo as a guest for interview. Bero is one of my very first listeners not directly related to me. We got acquainted after release of Episode 2. We have had a long mail-discussion about debugging, our different and common understanding. Especially Bero’s very interesting debugging tools made me curious.

As more as we discussed and as more as I got familiar with Bero, I more and more get the impression, that he’s Mr. Bug-Hunting in person. Today I have the pleasure to present you Bero Brekalo. We’re talking about many details of debugging. You get further details about his approach and his understanding of debugging. And you get acquainted with a very experienced engineer, who has gone through a lot of storms and bad weather. Let’s stay with me and enjoy the interview.

Everybody has his limit in handling emails. For one 10 per day are too much, another handles 150 without any problems. But sooner or later, everybody of us have to declare: it’s too much.

I have had a time in my life in which I have had to handle 750 mails per day in average. Nobody can say that this is possible to do. But that was the time I started a very strict and consequent filtering and categorization. However it still was too much. The combination of importance and urgency was back-breaking.

In the first part of this episode I will show you some of the most successful approaches how to handle the flood of emails daily arriving in your inbox. There are different concepts, sometimes even contradicting systems, but give them a try and find the most acceptable way for you.

But if you’re in the same situation as I was, even the most sophisticated approaches to handle the flood, weren’t succeeding, then you might be interested in the 2nd part of this episode.

This summer I stumbled over the book of Rory Vaden | Procrastinate on Purpose: 5 Permissions to Multiply Your Time. Rory was welcome as guest in the Art of Charm Podcast

His book was an eye-opener. I do not agree with everything Rory has written, however the major part, the introduction of a 3rd dimension, was essential. If importance and urgency are the first two dimensions, significance becomes the 3rd dimension.

You only can pass through time. Time simply goes on. It cannot be managed. The only thing that can be managed is you. Managing yourself is the only way you can change something significant in your email handling.

Importance means How much does this matter?. Urgency means How soon does this matter? Both are regularly orthogonal in its use. There could be made a matrix of importance to urgency relations. Further details could be find in this episode.

That’s highly individual. For one 20 mails per day are already too many. Another is happy to handle 200 without any problems. However there is a limit for everybody. And sooner or later you will hit this limit. Everybody faces the same problem of too many emails sooner or later. Listen to this episode to find out my new way how to handle emails more appropriately.

I have selected some of the most interesting and helpful hints from all over the place. Here are my preferrred action:

It’s not only the way how I handle emails, but how I handle all of my tasks. Taking significance into additional account, I now follow some very simple rules:

Taking importance and urgency into account let you get stuck in a horrible life movie. You are completely remote driven by the different requests from the outside. Change that to a third dimension and take significance into account. This will limit the impact of importance and urgency significantly.

Do you have ever heard about Soft-Errors? No, I do not mean Software-Errors. That’s something completely different. I mean Soft in difference to Hard. I first time get into contact with Soft-Errors in late 2011 as we have observed a spontaneous reset within a customer’s appliance. There was no indication for a reset-request, the system simply resetted.

Within some industries this is no case to be worried about. But in that time, in that type of industry, with this particular customer it was a challenge to find the root-cause of this spontaneous reset. And it was even more complicated to approve this root-cause.

In this episode it is Michal Lewczyk joining us. Michal is one of the very seldom specialists available for Soft-Errors. Michal was one of my colleagues in 2011 and the years after. He was the main investigator for all Soft-Error issues we suspected.

Michal has a deep knowledge in bare metal SW-programming. He’s mainly engaged into developing software for Digital Signal Processors; thus he’s not only an elaborated and experienced software-engineer, but also a great investigator and marvelous bug-hunter.

Let’s enjoy the engineer’s talk with Michal Lewczyk about one of the most quirky phenomenon in Embedded Systems.

With episode #6 the mini-series about task-forces has started. Episode #8 tackled the launch phase of a task-force. And with episode #12 you got a lot of details about the most essential part of a task-force – the running phase. And now, in this last episode of the series, it’s about all the stuff you need to do after you have run any task-force meeting or the task-force at all.

In the first step of the task-force mini-series you needed to understand the general pros and cons of a task-force, the intention and your own understanding of the why, what and how is essential.

In the second step I have informed you about all the valid key indicators for a successful launch phase. And I have given you the tools to do it.

The main phase of task-force, the essential doing, was covered in the third part. You found a lot of details about running the task-force, calling regular meetings, pushing forward, negotiations and arguments, and you strong commitment with a clear focus on the goals.

This episode now finalizes the series about task-force’ing. The aftermath covers the creation of meeting minutes, support of ongoing work and finally touch-down the task-force.

Since two weeks the Volkswagen scandal is swapping through the newspapers and the TV. Lots of details are already known, but tons of information are still unveiled. Step by step the responsibles are giving some light into the darkness of this threatening affair. An issue which might have the power for an earthquake in the German car-manufacturing industry.

In this episode I am talking about the technical details which triggered the current situation. I wanted to reflect the current problems without having any idea of solution. Considering the results which are day-by-day shown in the news we can conclude what might be going on behind the scenery.

BTW, I am sorry for any inconvenience with the audio of this episode. In the early versions I have not mono’fied the input, but you were bothered with the stereo version.

It currently looks like a big mess, however Volkswagen is a big and strong company and might dig its way out of its problems. However there are tons of issues, hundreds of technical challenges and millions or annoyed customers. I wanted to give you my perspective on to the situation – with all its actions, assumptions and believes.