Outline

In this short whitepaper we want to give a brief introduction on how automotive software engineering became as complex at it is today and what we need to do to improve the situation. We approach the topic from an organizational and tooling point of view.

First, we will have a look how the size of software changed in the recent years. Then we will show how not only the size but also the complexity increased at the example of the features of an ACC (adaptive cruise control). Third, we will see how currently such complex systems are developed. And finally, we will summarize the challenges and what needs to be done to mitigate them.

Historical view: why did software became much more complex?

Let us start our analysis with a historical view on software engineering.

Software has become much larger

In the beginning software was developed by very few highly qualified experts. These had the complete system design and code in their head. No coordination overhead, no information loss between different people, let alone departments companies.

Software has since then been continuously growing in size. See the illustration showing some numbers to get an impression on how much larger software has become.

For comparison, the bible would very roughly have a value of <0,8 (800.000k words with <10 words in a line.

Software systems consist basically oft two types of entities: processing and transport. Where the processing entities are strongly hierarchical and build very complex and time-bound networks. This might be comparable to a logistics network in manufacturing:

And with size comes the consistency problem. We know this from book or movies: suddenly something is written in a later chapter which does not fit to the beginning. In a book or movie, we usually do not notice this, and when we do, we ignore it. In a car software, this can lead to a crash.

Software systems have become much more complex

Why did the software become so much larger? Earlier software was more procedure and expert system style: the expert formalized their knowledge in form of code. The code was a procedure of what the system has to do exactly. Complemented with controllers this was enough to land on the moon and go back!

But these procedures could not assess their general situation and surrounding – this became possible only by the development of more advanced sensors over time. So, these procedures run their script and are – potentially catastrophically – stuck when the real world deviates from the assumption. Modern software in a car continuously analyses and assesses the environment around the car even if it is not directly needed for the main task. Based on this assessment the software can react to unforeseen circumstances. On top of that, the software is more and more able to be used by and to interact with untrained users. These can potentially do stupid things – the car software needs to stay save anyway!

Let’s have a look at the development stages of ACC – adaptive cruise control as an example:

Coming back to the comparison with the logistics system: while both being very complex, the logistics system has a lot of human experts keeping it running and intervening in cases of distortions. The software system has to manage all this by its own!

Current vehicle software systems for driving functions

Now let’s have a look more on how SW is structured. The exponential increase in complexity is not only coming from the individual functions as ACC which we have seen above. But also, the amount of functions increases massively, leading to basically comlexity³: [software module size] x [complexity of function] x [number of functions].

If we take the 100mio LOC from the car, this gives us approx. 2Mio software functions (rule of thumb: not more than 50 LOC in a function). With assumed 10 functions in a module and 10 modules in a group and 10 groups in a component we get very roughly 200k modules, 20k groups and 2k components (SWC).

In a modern premium car with many driving functions we find 150 or more Electronic Control Units (ECUs = “Computers”). Giving around 10 SWC per ECU – in average. The average has a wide spread – there are for example seat control units with only one SWC and complex driving functions with a significant 2-digit number. These ECUs, in turn, are connected by a large number of different vehicle-bus systems and must function as an entire time-synchronous network.

This is interesting as you can map this to organizational structures of companies: typically, one team is responsible for one group of modules.

A module is a portion of SW which realizes a certain task, like finding a face in a picture while an SWC realizes a complex feature like identifying and following a certain person.

How is such a complex system developed?

Now we are coming to the core of the problem: we have 100s of specialized development teams. Each knows it’s module and maybe driving function. These teams are split between two dozen companies, OEM and TIER-1 to -3. Alone in the OEM R&D you will find easily 3000 developers. And the same in each of the major TIER-1s.

How can this be coordinated?

Typically, there are a few Architects who design the system and shall have a perfect overview of the whole system and development states and interactions.

As the coordination is too huge, they are supplemented by complete teams of requirement engineers. A medium driving function can come up with 10.000 to 50.000 individual requirements. Requirements can be very specific like a mathematic formular or also abstract like “when a human is detected on the road the ego vehicle shall keep 1m distance and pass not faster than 30kmph”. This is an amount where writing on the OEM-side and analyzing on the supplier side becomes a project of its own. Now ingle person is able anymore to have a good and detailed overview.

To make it worse, architects are burdened with a lot of secondary coordination tasks. For example, the infamous “generation of the RTE” – which basically means the architect has to maintain the interfaces of 100s of modules and align all changes with the teams and then maintain these changes of 1000s of data specifications in often cumbersome tools.

Another key role shall come to the Function Owners. They shall understand and coordinate the development of individual modules. But also, these are burdened with a load of secondary coordination tasks, like creating development tickets for each change the Architect makes.

All this becomes even more problematic when you consider that in one single development path there can easily be 5 different companies involved, each with their IP protection in place.

Having a look at today’s tools

What are toolchains and which are there today?

When referring to “Toolchain” this usually simply means a list of different tools which are used subsequently during a development phase. These cover typically the areas of requirements, system architecture, software architecture, coding, integration, testing and development.

It cannot be said “which toolchains” are there today because they are collected from several different vendors and the number of possible combinations is huge. Even large well-known vendors only cover a small slice of the needed tools.

What is AUTOSAR? And what are AUTOSAR Toolchains?

AUTOSAR is foremost a specification on how the software components shall run on a hardware and interact with each other. When saying “AUTOSAR” often not the specification is meant, but one of the implementations by one of the well-known vendors. In this case it is a software framework in which the software modules are integrated and which are then in a defined process made to run on the target hardware.

The strength of AUTOSAR is that it works for hundreds of companies.

The weakness of AUTOSAR is that a lot of companies contributed to the development and the resulting specification is a huge jack-of-all-traits which does not focus on the real intent but was meant to fulfill the wishes of all contributors.

What is ROS by the way?

ROS is basically the opposite: it is also a framework for (robot) module integration, but it is community driven and the main focus is the quick and easy realization of software.

The disadvantage is that it is not optimized for being light in hardware resource consumption and functional safety and determinism are not a concern.

Summarizing the challenges

Finally, we will summarize the main challenges from a tooling and organizational point of view. Issues here lead to issues in other areas. For instance, if an overloaded Architect makes – or does not detect – a contradiction in the architecture this will result in several bugs on different levels later.

Gaps between people

Huge amounts of people have to work together to achieve this task and manage the complexity. But these people have a limited mental and temporal capacity. These people, namely Architects, Function Owners, Requirement Engineers, Test Managers, FUSI Managers currently invest big shares of their time in secondary coordination tasks like distributing information, chasing down lost information, discussing interfaces and worst of all: putting data into cumbersome tools.

• They have to be supported by tools to remove the burden of secondary coordination tasks so that they can really focus on their critical mission: managing the massively complex software system.

Tool efficiency

The above situation is made worse by the fact that a lot of these tools have inherent shortcomings of two sorts:

Process vs. usability

The tools are often design do perfectly reproduce a formal process like V-Modell and Autosar. These tools are perfectly fit to let the user tweak each small nuance of the huge Autosar spec. But 90% of this is not needed in daily work.

• Tools are needed which focus on the actual lived development interaction, not focused on a process by the book. They need to focus on efficiency and usability to release exactly the key people from their burden of maintaining and aligning huge amounts of data manually.

Inconsistent tool chains

The tools used are usually collected from several different vendors and are made for several different purposes which are not aligned. In the result, data exchange is often done by exporting and importing data, email, Excel sheets, some home brewed scripts and a lot of manual work. Often the same data has to consistently be maintained in several tools.

• The integration of tools has to improve. We need a tool-chain out of one mold. It will not come from one vendor, but nevertheless, the tools need to be integrated smoothly, so that exactly the key coordinating people who work with a lot of tools to not waste their valuable effort.

Gaps between pieces of technology: remote hardware

And finally, a challenge is the special distribution of these 100s development teams over half of the world. An ECU is developed in on place, the base software in a second and the driving functions in a third. Now a tester finds a bug in this piece of hardware on their desk. Currently, it is a big issue that the different teams cannot access the faulty hardware. Similar situations occurs with test vehicles on the road. A lot of project time s wasted by incorrectly recorded data, just because a software version or one parameter of the 100s of modules was incorrect and nobody was able to check this while the car was on the road.

• The whole development platform needs to be cloud-based so that everybody can access every relevant piece of information, independent from their physical location. Furthermore, each piece of hardware must be cloud-enabled so that it can be accessed. At least during the development phase.

Conclusion

When talking about autonomous driving, usually the technical challenges and achievements are discussed. But on the tooling and process side the challenges are similarly strong – and in case of failure maybe even more critical. A technical issue can be tested, identified and fixed. An organizational issue is a continuous source of errors.