Integration of Performance Management into the Application Lifecycle

MA-Thesis / Master von Eduard Tudenhöfner

Introduction:

Despite the widespread recognition that performance is important to the success of a project, many software products fail to respond fast enough to user requests or to handle a certain amount of parallel business transactions. This is because nowadays projects are result-oriented where the focus is on functionality to be implemented. Such projects do not pay high attention to application performance because it still does not have the particular importance that unit testing for example has. Moreover today’s development models usually consider performance management only in a limited way within their lifecycle and often follow what is known as the fix it later approach. The fix it later approach concentrates on software correctness and defers performance considerations to the integration testing phase where additional hardware is added or a system is tuned when performance issues are detected.

The problem of neglecting performance management is that performance issues often do not emerge until an application is put into production, where it is likely to suffer the consequences of a performance failure. The consequences of performance failures can be increased operational costs, increased development and hardware costs, and damaged customer relations. If severe performance issues are discovered during production, it may be too expensive to re-design a system or even impossible to add additional hardware in order to meet performance objectives. Such projects are likely to be canceled and their costs will be unrecoverable.

To avoid such situations, performance management should be integrated into an application’s lifecycle from the beginning. This means that performance objectives have to be defined early within a project and continually verified as an application evolves. Having performance management integrated from the beginning allows to reduce overall project risk and costs because performance issues can be spotted and corrected early in the lifecycle and even before end users are affected. Furthermore an application is extensively tested for its ability of reaching performance objectives before it is deployed to a production environment and exposed to real users.

NovaTec GmbH is a company providing IT-services in the area of consulting, project management, software engineering, application architectures, provisioning, performance management, and process engineering. The competences of NovaTec are logically grouped in so-called competence areas. This thesis is written in the competence area Application Performance Management, where the purpose is to support customers in performance engineering and analysis tasks in order to detect performance bottlenecks and stability issues. Additionally, this competence area helps customers in introducing performance management to their projects.

Introducing performance management to customers is currently solely based on the experience of NovaTec experts. This experience is only partially documented and difficult to reuse. The problem is that no model is used for guiding the introduction of performance management to different types of projects. For that reason a well-documented performance management model is required that includes the experience of experts and is generally applicable to different development models.

As there is currently no model available for guiding performance management, the objectives of this thesis are as follows:

1) Evaluation of Existing Approaches Existing approaches towards performance management have to be analyzed and evaluated in how far they are suitable for nowadays project circumstances.

2) Performance Activities and their Responsibilities It must be analyzed which performance activities are to be considered in an application’s lifecycle and by whom they have to be performed. Furthermore it has to be evaluated which performance artefacts are necessary and how they are related to particular performance activities.

3) Analysis of Development Models As nowadays development models are not considering performance management, two development models have to be analyzed in order to determine how far they already support performance considerations and how they can be enhanced for performance management.

4) Creation A performance management approach should be generally applicable for different development models, thus if the outcome of the analysis tasks indicate that an available approach is suitable for nowadays projects, it should be selected and adapted to the needs of NovaTec. Otherwise, an own approach that concludes the analysis results has to be created and documented.

5) Application Example As a final step, the performance management approach should be combined with one development model in order to show an example of how the approach can be used.

The outcome of this master thesis serves as a starting point for a documented performance management approach that is used by NovaTec experts for introducing performance management to customer projects.

This thesis is based on research that was done in the area of Software Performance Engineering (SPE), which is often used in the literature interchangeably with the term Application Performance Management (APM). Based on the literature, both disciplines encompass management and engineering activities, roles, and practices that are performed throughout an application’s lifecycle. Nevertheless, the competence area Application Performance Management differentiates between performance engineering and performance management. Performance engineering is about the technical aspect when executing performance activities, whereas performance management covers performance engineering activities and additionally comprises management activities. Due to the fact that most of the analyzed literature did not make any difference between SPE and APM, the second and third chapter will use both terms interchangeably in order to not falsify an author’s thoughts. Afterwards the terms APM and performance management will be used consequently until the end of the thesis.

Chapter 2 describes Software Performance Engineering in general and shows how performance can be quantified. The problems of performance engineering are demonstrated and the differences between reactive and proactive performance management are explained. Chapter 3 analyzes a process towards performance engineering and evaluates which performance activities can be used in nowadays projects. Additionally, performance roles and artefacts are depicted and discussed in-depth. Moreover two different development models are analyzed for their suitability of performance integration.

Chapter 4 depicts the selected solution approach and describes its structure and performance activities. Furthermore defined performance roles and artefacts with their interdependencies are specified.

Chapter 5 applies the selected approach and integrates it into a development model in two different ways. The first way handles a best-case scenario, where performance management is integrated from the beginning of a project. The second way deals with an integration after performance issues were found.

Chapter 6 reviews the results of this thesis and describes learned lessons. Additionally, an outlook for future work is provided.

Table of Contents:

Text Sample:

Chapter 4, Application Performance Management Model:

This chapter describes the designed APM model and its components based on previous analysis results.

4.1, Overview:

The purpose of an own performance management approach is that it is generally applicable to different development models. For that reason a sequential proceeding on a performance-activity-level, such as the SPE process has it, is inappropriate due to several drawbacks. Another way would be to define a general proceeding of performance management in order to have a higher flexibility. This can be done by dividing the entire proceeding into four main parts. The purpose of the first part would be to lay the groundwork for performance management by planning performance activities and building up a performance team. Afterwards the performance could be predicted by defining performance objectives and initially verified through architectural performance considerations in the second part. These objectives could then further be verified through performance testing in the third part. The last part would cover the validation of performance objectives in a production environment through monitoring. Additionally, this part would use forecasting in order to predict how an application will behave in future.

These four parts are defined as the building blocks of the APM model. Having such building blocks allows to apply the APM model to development models that are either phase-oriented or stage-oriented because building blocks are only a logical grouping of activities and are independent of phase-oriented or stage-oriented approaches. In addition to that, the flexibility of the APM model is increased because it can be tailored to project needs and therefore adapted in different ways. The tailoring capabilities of the APM model are described in section 4.6. Figure 4.1 shows the generic APM model with its building blocks. Each building block is described in detail in section 4.2.

These building blocks define the overall flow of performance management. Depending on the development model where the APM model is applied, it is performed either within each project phase or each iteration. Figure 4.2 shows an example of how the APM model is mapped to a development model that has phases which are further divided to iterations. In the first phase the APM model is performed once because the first phase is finished after one iteration. The APM model is applied twice during the second phase because two iterations are needed to finish this phase. As the third phase is completed after three iterations, the APM model is performed thrice. The fourth phase is similar to the second phase, where the APM model is applied twice due to two iterations.

4.2, Building Blocks:

The building blocks comprise the entire set of performance activities that are selected from section 3.1.1 and section 3.2, thus each block of the APM model is broken down to its performance activities, which are illustrated in figure 4.3. The numbering within each block specifies a certain order in which the activities can be performed. Yet, it is also possible to perform activities in a different order or concurrently if project circumstances require it. In this case the APM model does not restrict the usability of performance management. For example, there is no contradiction if performance risks are identified and performance objectives are specified at the same time. Nevertheless, some performance activities strictly depend on other activities that must be performed in advance. An example depicting a performance activity with its dependencies is described in section 4.3.2. The entire set of activity dependencies can be found in Confluence.

Due to the fact that not every performance activity is performed in each project phase or iteration, the APM model varies in its level of execution. For example, it is not useful to apply performance testing at the beginning of a project where no executable is available. The level of execution consists of the scope of a block and the level of detail of an associated performance activity. The scope of a block is defined by the number of activities that are included in this block. The level of detail of an activity is defined by the work that is done within this activity. Figure 4.4 shows an example, where the scope of a building block is reduced to four performance activities.

In order to provide an easy understanding and to define clear responsibilities, each performance activity is further broken down to show executive roles with associated artifacts. Section 4.4 describes the defined performance roles and artifacts in depth. The following sections describe each building block and its performance activities. As the purpose of the APM model is not to describe every performance activity in detail, the following performance activities are only generally described.

4.2.1, Planning Performance Management:

The first block of the APM model is about planning performance and consists of four performance activities. These activities are a prerequisite for the activities of the other building blocks. For example, it should be first planned when performance testing is to be done before it is carried out. The following sections describe the associated activities of the first block in detail.

4.2.1.1, Plan Performance Management:

It must be planned when certain performance activities should be performed and what their contents are. It is important to feed back this into the project plan in order to know when certain activities are to be executed and by whom, otherwise there is a chance that performance activities are omitted. For example, time for performance testing has to be considered within the project plan and therefore must be planned in advance.

4.2.1.2, Build up Performance Team:

It is essential to establish a performance team that is responsible for performance management. Depending on the project size and risk, such a team can consist of one or more individuals. The creation of a performance team should be done when the APM model is introduced to a project. The roles and responsibilities of the performance team are depicted in section 4.4.1.