484:
387:
point the system will no longer meet the criteria specified within the service level agreements. Capacity management is charged with ensuring that additional capacity is added in advance of that point (additional CPUs, more memory, new database indexing, et cetera) so that the trend lines are reset and the system will remain within the specified performance range.
25:
374:
requirements. Transaction response time is logged in a database such that queries and reports can be run against the data. This permits trend analysis that can be useful for capacity management. When user transactions fall out of band, the events should generate alerts so that attention may be applied to the situation.
215:
are identified. Typically they are classified as critical based upon revenue value, cost savings, or other assigned business value. This classification is done by the business unit, not the IT organization. High level risks that may impact system performance are identified and described at this time.
386:
on historical monitoring generated data, such that the future time of non compliance is predictable. For example, if a system is showing a trend of slowing transaction processing (which might be due to growing data set sizes, or increasing numbers of concurrent users, or other factors) then at some
302:
If performance engineering has been properly applied at each iteration and phase of the project to this point, hopefully this will be sufficient to enable the system to receive performance certification. However, if for some reason (perhaps proper performance engineering working practices were not
422:
The trend analysis component of this cannot be undervalued. This functionality, properly implemented, will enable predicting when a given application undergoing gradually increasing user loads and growing data sets will exceed the specified non functional performance requirements for a given use
395:
Within the problem management domain, the performance engineering practices are focused on resolving the root cause of performance related problems. These typically involve system tuning, changing operating system or device parameters, or even refactoring the application software to resolve poor
127:
As the connection between application success and business success continues to gain recognition, particularly in the mobile space, application performance engineering has taken on a preventive and perfective role within the software development life cycle. As such, the term is typically used to
235:
The type of requirements that relate to performance engineering are the non-functional requirements, or NFR. While a functional requirement relates to which business operations are to be performed, a performance related non-functional requirement will relate to how fast that business operation
404:
To ensure that there is proper feedback validating that the system meets the NFR specified performance metrics, any major system needs a monitoring subsystem. The planning, design, installation, configuration, and control of the monitoring subsystem are specified by an appropriately defined
373:
and the associated systems monitoring that serves to validate service level compliance, detect problems, and identify trends. For example, when real user monitoring is deployed it is possible to ensure that user transactions are being executed in conformance with specified non-functional
303:
applied) there are tests that cannot be tuned into compliance, then it will be necessary to return portions of the system to development for refactoring. In some cases the problem can be resolved with additional hardware, but adding more hardware leads quickly to diminishing returns.
135:
encompasses more than just the software and supporting infrastructure, and as such the term performance engineering is preferable from a macro view. Adherence to the non-functional requirements is also validated post-deployment by monitoring the production systems. This is part of
278:
The performance test team normally does not execute performance tests in the development environment, but rather in a specialized pre-deployment environment that is configured to be as close as possible to the planned production environment. This team will execute
216:
An example might be known performance risks for a particular vendor system. Finally, performance activities, roles and deliverables are identified for the
Elaboration phase. Activities and resource loading are incorporated into the Elaboration phase project plans.
147:
Performance engineering has become a separate discipline at a number of large corporations, with tasking separate but parallel to systems engineering. It is pervasive, involving people from multiple organizational units; but predominantly within the
267:
Identify a database test data load tool for the development/component unit test environment; this is required to ensure that the database optimizer chooses correct execution paths and to enable reinitializing and reloading the database as
315:
Configuring the operating systems, network, servers (application, web, database, load balancer, etc.), and any message queueing software according to the base checklists and the optimizations identified in the performance test
423:
case. This permits proper management budgeting, acquisition of, and deployment of the required resources to keep the system running within the parameters of the non functional performance requirements.
128:
describe the processes, people and technologies required to effectively test non-functional requirements, ensure adherence to service levels and optimize application performance prior to deployment.
646:
258:
Specify an automated unit (component) performance test tool for the development/component unit test environment; this is used when no GUI yet exists to drive the components under development.
50:
264:
Specify an automated multi-user capable script-driven end-to-end tool for the development/component unit test environment; this is used to execute screen-driven use cases.
204:
Because this discipline is applied within multiple methodologies, the following activities will occur within differently specified phases. However, if the phases of the
596:
710:
631:
228:. Probe cases will be decomposed further, as needed, to single page (screen) transitions. These are the use cases that will be subjected to script driven
683:
330:
Validating that weekly and monthly performance reports indicate that critical use cases perform within the specified non functional requirement criteria
382:
For capacity management, performance engineering focuses on ensuring that the systems will remain within performance compliance. This means executing
1112:
753:
501:
418:
It enables the ability to track trends over time, such as the impact of increasing user loads and growing data sets on use case level performance.
35:
311:
During this final phase the system is deployed to the production environment. A number of preparatory steps are required. These include:
117:
548:
703:
606:
567:
520:
68:
668:
299:. Where necessary, the system will be tuned to bring nonconforming tests into conformance with the non-functional requirements.
527:
164:
Eliminate system failure requiring scrapping and writing off the system development effort due to performance objective failure
46:
1117:
870:
763:
738:
505:
85:
659:
829:
743:
534:
1091:
696:
295:
that will identify the system bottlenecks. The data gathered, and the analysis, will be fed back to the group that does
280:
261:
Specify an automated tool for driving server-side unit (components) for the development/component unit test environment.
229:
205:
906:
819:
252:
349:
In the operational domain (post production deployment) performance engineering focuses primarily within three areas:
287:, validating that the critical use cases conform to the specified non-functional requirements. The team will execute
516:
768:
89:
611:
494:
916:
901:
679:
Performance
Evaluation of an Air Traffic Control System using the Application Response Measurement (ARM) Standard
292:
931:
809:
804:
748:
778:
370:
1053:
758:
149:
987:
957:
733:
161:
Increase business revenue by ensuring the system can process transactions within the requisite timeframe
137:
97:
412:
It is possible to turn on and turn off monitoring at periodic points or to support problem resolution.
1078:
1048:
626:
541:
180:
121:
1058:
1043:
1012:
719:
601:
109:
1068:
1063:
814:
783:
616:
296:
244:
Early in this phase a number of performance tool related activities are required. These include:
336:
Identify projected trends from monthly and quarterly reports, and on a quarterly basis, execute
186:
Reduce increased software maintenance costs due to software impacted by ad hoc performance fixes
326:
Once the new system is deployed, ongoing operations pick up performance activities, including:
1022:
962:
651:
337:
189:
Reduce additional operational overhead for handling system issues due to performance problems
1027:
997:
926:
921:
860:
845:
471:"Banking Industry Lessons Learned in Outsourcing Testing Services," Gartner. August 2, 2012.
447:
442:
432:
212:
274:
Presentations and training must be given to development team members on the selected tools.
1073:
972:
967:
911:
886:
850:
663:
452:
101:
678:
592:
Practical
Performance Analyst - Performance Engineering Community & Body Of Knowledge
621:
1002:
992:
947:
891:
788:
383:
248:
Identify key development team members as subject matter experts for the selected tools.
224:
During this defining phase, the critical business processes are decomposed to critical
291:
against a normally expected (median) load as well as a peak load. They will often run
1106:
982:
977:
952:
824:
586:
1017:
288:
656:
483:
437:
369:
In the service level management area, performance engineering is concerned with
1007:
673:
591:
322:
Running statistics on the database after the production data load is completed
93:
641:
284:
409:
It is possible to establish service level agreements at the use case level.
652:
The
Vicious Cycle of Computer Systems Performance and IT Operational Costs
208:(RUP) are used as a framework, then the activities will occur as follows:
16:
Encompasses the techniques applied during a systems development life cycle
636:
225:
865:
319:
Ensuring all performance monitoring software is deployed and configured
688:
855:
211:
During the first, Conceptual phase of a program or project, critical
333:
Where use cases are falling outside of NFR criteria, submit defects
637:
Performance and
Scalability of Distributed Software Architectures
896:
358:
354:
350:
141:
692:
674:
Performance
Testing Web Services: Strategies and Best Practices
477:
18:
176:
Avoid additional and unnecessary hardware acquisition costs
170:
Eliminate avoidable system rework due to performance issues
104:
usage) will be met. It may be alternatively referred to as
167:
Eliminate late system deployment due to performance issues
255:
tool for the development/component unit test environment.
396:
performance due to poor design or bad coding practices.
192:
Identify future bottlenecks by simulation over prototype
42:
622:
Introduction to
Modeling Based Performance Engineering
271:
Deploy the performance tools for the development team.
1036:
940:
879:
838:
797:
726:
642:
508:. Unsourced material may be challenged and removed.
627:Leveraging ITIL to Improve Application Performance
405:monitoring process. The benefits are as follows:
647:Software Engineering and Performance: A Road-map
632:Patterns & Practices Performance Engineering
183:costs due to performance problems in production
704:
684:Integration of Performance Management in ITIL
415:It enables the generation of regular reports.
8:
84:encompasses the techniques applied during a
711:
697:
689:
657:Microsoft Windows Server Performance Team
617:Exploring UML for Performance Engineering
568:Learn how and when to remove this message
173:Eliminate avoidable system tuning efforts
69:Learn how and when to remove this message
754:Earth systems engineering and management
464:
236:performs under defined circumstances.
7:
607:A Performance Process Maturity Model
506:adding citations to reliable sources
36:research paper or scientific journal
612:The Every Computer Performance Book
597:Performance Engineering Methodology
118:application performance engineering
669:Gathering Performance Requirements
602:A Performance Engineering Strategy
156:Performance engineering objectives
14:
789:Sociocultural Systems Engineering
587:Database Performance Tuning Guide
482:
200:Performance engineering approach
114:software performance engineering
23:
1113:Software performance management
493:needs additional citations for
106:systems performance engineering
871:Systems development life cycle
764:Enterprise systems engineering
739:Biological systems engineering
86:systems development life cycle
1:
830:System of systems engineering
744:Cognitive systems engineering
907:Quality function deployment
820:Verification and validation
90:non-functional requirements
1134:
769:Health systems engineering
195:Increase server capability
1087:
917:Systems Modeling Language
517:"Performance engineering"
92:for performance (such as
932:Work breakdown structure
810:Functional specification
805:Requirements engineering
749:Configuration management
371:service level agreements
365:Service level management
351:service level management
206:rational unified process
51:overly technical phrases
43:help improve the article
779:Reliability engineering
774:Performance engineering
133:performance engineering
82:Performance engineering
1054:Industrial engineering
759:Electrical engineering
150:information technology
1118:Software optimization
988:Arthur David Hall III
958:Benjamin S. Blanchard
734:Aerospace engineering
340:management activities
138:IT service management
1079:Software engineering
1049:Computer engineering
502:improve this article
181:software maintenance
122:software engineering
1059:Operations research
1044:Control engineering
1013:Joseph Francis Shea
720:Systems engineering
378:Capacity management
355:capacity management
281:performance testing
230:performance testing
110:systems engineering
45:by rewriting it in
1069:Quality management
1064:Project management
892:Function modelling
815:System integration
784:Safety engineering
662:2010-05-04 at the
391:Problem management
359:problem management
345:Service management
297:performance tuning
213:business processes
47:encyclopedic style
34:is written like a
1100:
1099:
1023:Manuela M. Veloso
963:Wernher von Braun
578:
577:
570:
552:
338:capacity planning
179:Reduce increased
79:
78:
71:
1125:
1028:John N. Warfield
998:Robert E. Machol
927:Systems modeling
922:Systems analysis
861:System lifecycle
846:Business process
713:
706:
699:
690:
573:
566:
562:
559:
553:
551:
510:
486:
478:
472:
469:
448:Software testing
443:Software quality
433:Java performance
74:
67:
63:
60:
54:
27:
26:
19:
1133:
1132:
1128:
1127:
1126:
1124:
1123:
1122:
1103:
1102:
1101:
1096:
1083:
1074:Risk management
1032:
973:Harold Chestnut
968:Kathleen Carley
936:
912:System dynamics
887:Decision-making
875:
851:Fault tolerance
834:
793:
722:
717:
664:Wayback Machine
583:
581:Further reading
574:
563:
557:
554:
511:
509:
499:
487:
476:
475:
470:
466:
461:
453:Web performance
429:
402:
393:
380:
367:
347:
309:
242:
222:
202:
158:
75:
64:
58:
55:
40:
28:
24:
17:
12:
11:
5:
1131:
1129:
1121:
1120:
1115:
1105:
1104:
1098:
1097:
1095:
1094:
1088:
1085:
1084:
1082:
1081:
1076:
1071:
1066:
1061:
1056:
1051:
1046:
1040:
1038:
1037:Related fields
1034:
1033:
1031:
1030:
1025:
1020:
1015:
1010:
1005:
1003:Radhika Nagpal
1000:
995:
993:Derek Hitchins
990:
985:
980:
975:
970:
965:
960:
955:
950:
948:James S. Albus
944:
942:
938:
937:
935:
934:
929:
924:
919:
914:
909:
904:
899:
894:
889:
883:
881:
877:
876:
874:
873:
868:
863:
858:
853:
848:
842:
840:
836:
835:
833:
832:
827:
822:
817:
812:
807:
801:
799:
795:
794:
792:
791:
786:
781:
776:
771:
766:
761:
756:
751:
746:
741:
736:
730:
728:
724:
723:
718:
716:
715:
708:
701:
693:
687:
686:
681:
676:
671:
666:
654:
649:
644:
639:
634:
629:
624:
619:
614:
609:
604:
599:
594:
589:
582:
579:
576:
575:
490:
488:
481:
474:
473:
463:
462:
460:
457:
456:
455:
450:
445:
440:
435:
428:
425:
420:
419:
416:
413:
410:
401:
398:
392:
389:
384:trend analysis
379:
376:
366:
363:
346:
343:
342:
341:
334:
331:
324:
323:
320:
317:
308:
305:
276:
275:
272:
269:
265:
262:
259:
256:
249:
241:
238:
221:
218:
201:
198:
197:
196:
193:
190:
187:
184:
177:
174:
171:
168:
165:
162:
157:
154:
152:organization.
88:to ensure the
77:
76:
31:
29:
22:
15:
13:
10:
9:
6:
4:
3:
2:
1130:
1119:
1116:
1114:
1111:
1110:
1108:
1093:
1090:
1089:
1086:
1080:
1077:
1075:
1072:
1070:
1067:
1065:
1062:
1060:
1057:
1055:
1052:
1050:
1047:
1045:
1042:
1041:
1039:
1035:
1029:
1026:
1024:
1021:
1019:
1016:
1014:
1011:
1009:
1006:
1004:
1001:
999:
996:
994:
991:
989:
986:
984:
983:Barbara Grosz
981:
979:
978:Wolt Fabrycky
976:
974:
971:
969:
966:
964:
961:
959:
956:
954:
953:Ruzena Bajcsy
951:
949:
946:
945:
943:
939:
933:
930:
928:
925:
923:
920:
918:
915:
913:
910:
908:
905:
903:
900:
898:
895:
893:
890:
888:
885:
884:
882:
878:
872:
869:
867:
864:
862:
859:
857:
854:
852:
849:
847:
844:
843:
841:
837:
831:
828:
826:
825:Design review
823:
821:
818:
816:
813:
811:
808:
806:
803:
802:
800:
796:
790:
787:
785:
782:
780:
777:
775:
772:
770:
767:
765:
762:
760:
757:
755:
752:
750:
747:
745:
742:
740:
737:
735:
732:
731:
729:
725:
721:
714:
709:
707:
702:
700:
695:
694:
691:
685:
682:
680:
677:
675:
672:
670:
667:
665:
661:
658:
655:
653:
650:
648:
645:
643:
640:
638:
635:
633:
630:
628:
625:
623:
620:
618:
615:
613:
610:
608:
605:
603:
600:
598:
595:
593:
590:
588:
585:
584:
580:
572:
569:
561:
550:
547:
543:
540:
536:
533:
529:
526:
522:
519: –
518:
514:
513:Find sources:
507:
503:
497:
496:
491:This article
489:
485:
480:
479:
468:
465:
458:
454:
451:
449:
446:
444:
441:
439:
436:
434:
431:
430:
426:
424:
417:
414:
411:
408:
407:
406:
399:
397:
390:
388:
385:
377:
375:
372:
364:
362:
360:
356:
352:
344:
339:
335:
332:
329:
328:
327:
321:
318:
314:
313:
312:
306:
304:
300:
298:
294:
290:
286:
282:
273:
270:
266:
263:
260:
257:
254:
250:
247:
246:
245:
239:
237:
233:
231:
227:
219:
217:
214:
209:
207:
199:
194:
191:
188:
185:
182:
178:
175:
172:
169:
166:
163:
160:
159:
155:
153:
151:
145:
143:
139:
134:
129:
125:
123:
119:
115:
111:
107:
103:
99:
95:
91:
87:
83:
73:
70:
62:
59:November 2016
52:
49:and simplify
48:
44:
38:
37:
32:This article
30:
21:
20:
1018:Katia Sycara
902:Optimization
773:
564:
555:
545:
538:
531:
524:
512:
500:Please help
495:verification
492:
467:
421:
403:
394:
381:
368:
348:
325:
310:
301:
293:stress tests
289:load testing
277:
243:
240:Construction
234:
223:
210:
203:
146:
132:
130:
126:
113:
105:
81:
80:
65:
56:
33:
438:Scalability
316:environment
220:Elaboration
1107:Categories
1008:Simon Ramo
558:March 2009
528:newspapers
459:References
400:Monitoring
307:Transition
285:test cases
251:Specify a
140:(see also
94:throughput
798:Processes
727:Subfields
253:profiling
226:use cases
131:The term
1092:Category
839:Concepts
660:Archived
427:See also
283:against
866:V-Model
542:scholar
268:needed.
120:within
108:within
98:latency
41:Please
941:People
856:System
544:
537:
530:
523:
515:
357:, and
112:, and
102:memory
880:Tools
549:JSTOR
535:books
100:, or
897:IDEF
521:news
142:ITIL
504:by
144:).
116:or
1109::
361:.
353:,
232:.
124:.
96:,
712:e
705:t
698:v
571:)
565:(
560:)
556:(
546:·
539:·
532:·
525:·
498:.
72:)
66:(
61:)
57:(
53:.
39:.
Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.