Non-functional Testing Results

From Evolution

The Non-functional Testing Results would be recorded for the following 3 categories:

  1. Mailer
  2. Calendar
  3. Address Book


Contents

Mailer

Scenario 1

The account type used for test execution was IMAP on a test IMAP server.

This test scenario consisted of the following 6 major operations in the evolution mailer:

  • Send Mail [With atleast 1 attachment]
  • Read Mail
  • Delete Mail
  • Copy Mail
  • Move Mail
  • Sort Mails

An input file with the number of times each of these above operations are to be performed is provided as an input to the test script. The order in which these operations are performed is randomized inside the test script.

Number of times the above mailer operations are performed were: Read Mail = 10 Send Mail = 20 Delete Mails = 20 Move mails = 10 Copy Mails = 25 Sort mails = 5

The statistics captured during the execution of the test script are mainly the Operation, Time, CPU Utilization and Memory usage (Resident) of the evolution process.

The tables shown below summarize the results

send_email

memory usage of "evolution" process when running send_email
Enlarge
memory usage of "evolution" process when running send_email
This graph shows the memory usage of the "evolution" client process when it is sending emails (with attachments)
Operation Time CPU Utilization (%) Resident Memory (in MB) Additional Information
send_email15:40:158.034Entered send_email routine
send_email15:43:290.040After typing the text in the compose message
send_email15:43:3711.937During verification of sent email
send_email15:43:400.037After sending and verifying the email
send_email15:49:164.038The above comments repeat for the remaining entries in send_email table as 4 readings correspond to 1 sent email
send_email15:50:400.042
send_email15:50:4727.139
send_email15:50:500.039
send_email15:56:540.040
send_email16:00:060.044
send_email16:00:142.042
send_email16:00:170.042
send_email16:00:530.042
send_email16:02:180.046
send_email16:02:260.042
send_email16:02:290.042
send_email16:04:180.042
send_email16:05:430.046
send_email16:05:5141.742
send_email16:05:540.042
send_email16:06:180.042
send_email16:09:310.046
send_email16:09:390.043
send_email16:09:410.043
send_email16:11:540.043
send_email16:15:120.046
send_email16:15:209.943
send_email16:15:230.043
send_email16:15:470.043
send_email16:19:110.047
send_email16:19:1911.944
send_email16:19:210.044
send_email16:20:280.045
send_email16:21:460.046
send_email16:21:540.045
send_email16:21:560.045
send_email16:22:230.045
send_email16:25:370.050
send_email16:25:450.046
send_email16:25:480.046
send_email16:25:540.046
send_email16:29:040.050
send_email16:29:120.046
send_email16:29:150.046
send_email16:29:210.046
send_email16:30:470.050
send_email16:30:5533.946
send_email16:30:580.046
send_email16:31:040.046
send_email16:33:440.050
send_email16:33:520.047
send_email16:33:550.047
send_email16:34:010.046
send_email16:35:320.049
send_email16:35:400.049
send_email16:35:430.049
send_email16:37:190.046
send_email16:40:312.050
send_email16:40:390.046
send_email16:40:420.046
send_email16:40:480.046
send_email16:42:120.050
send_email16:42:1930.246
send_email16:42:220.046
send_email16:42:490.046
send_email16:46:040.050
send_email16:46:1211.946
send_email16:46:150.046
send_email16:47:020.046
send_email16:50:140.050
send_email16:50:228.046
send_email16:50:240.046
send_email16:51:120.047
send_email16:52:340.051
send_email16:52:4242.047
send_email16:52:450.047
send_email16:53:570.047
send_email16:55:250.051
send_email16:55:3245.748
send_email16:55:350.048

read_email

evolution memory usage when reading emails
Enlarge
evolution memory usage when reading emails
This graph depicts the memory usage of the "evolution" client process when it is executing read_email.
Operation Time CPU Utilization (%) Resident Memory (in MB) Additional Information
read_email15:39:280.034
read_email15:43:460.037
read_email15:44:330.037
read_email15:45:500.038
read_email15:55:140.040
read_email15:56:240.040
read_email16:00:230.042
read_email16:02:350.042
read_email16:03:050.042
read_email16:09:470.043

copy_email

evolution memory usage when it is copying mails from one folder to another
Enlarge
evolution memory usage when it is copying mails from one folder to another

Operation Time CPU Utilization (%) Resident Memory (in MB) Additional Information
copy_email15:37:450.033
copy_email15:39:580.034
copy_email15:44:150.037
copy_email15:45:320.038
copy_email15:52:080.039
copy_email15:53:0518.137
copy_email15:53:320.039
copy_email15:54:360.039
copy_email15:54:550.040
copy_email16:03:340.042
copy_email16:06:000.042
copy_email16:10:180.043
copy_email16:10:360.043
copy_email16:19:480.044
copy_email16:35:492.046
copy_email16:36:0610.046
copy_email16:36:334.046
copy_email16:37:010.046
copy_email16:50:300.046
copy_email16:52:510.047
copy_email16:53:080.047
copy_email16:55:410.048
copy_email16:56:044.048
copy_email16:56:290.048
copy_email16:56:4614.048

move_email

evolution memory usage when it is moving mails
Enlarge
evolution memory usage when it is moving mails

Operation Time CPU Utilization (%) Resident Memory (in MB) Additional Information
move_email15:37:260.033
move_email15:48:010.038
move_email15:48:184.038
move_email15:50:560.039
move_email15:52:280.039
move_email15:52:460.039
move_email15:56:060.040
move_email16:03:5210.042
move_email16:11:360.043
move_email16:15:290.043

sort_email

evolution memory usage when sorting emails
Enlarge
evolution memory usage when sorting emails

Operation Time CPU Utilization (%) Resident Memory (in MB) Additional Information
sort_email15:45:020.037
sort_email15:46:410.038
sort_email15:47:310.038
sort_email15:48:462.038
sort_email15:51:3841.939

Scenario 2

This test scenario consisted only the send email operation continuously with an IMAP account and observe the memory usage of evolution client. Only plain text emails were sent in this scenario and at specific counts of sent emails the memory usage of evolution was noted.

The following 2 graphs summarizes the outcome.


"evolution" Memory usage
Enlarge
"evolution" Memory usage
This graph shows the memory consumption by the main evolution client process.
 Other evolution processes Memory usage
Enlarge
Other evolution processes Memory usage
This graph shows the memory consumption of other evolution processes i.e. evolution-data-server, evolution-exchange-storage and evolution-alarm-notify.

Observations/Concerns

I have made following observations after executing the above mentioned test scenarios:

PS: Some of these might qualify as bugs and can be filed in evolution bugzilla Need Comments

  1. The memory usage of evolution process grows continuously over a period of time.
  2. The CPU utilization is constant most of the times at 0.0 but in between some of the operations (sort, copy, send and move emails) it shoots up once. The maximum CPU utilization seen in this test execution is 47%. As the number of operations and the time period of execution increase this value is seen to be touching 99.9% at one point of time.
  3. As the number of operations increase the Memory usage keeps on growing and doesn't come down even if we are not using the evolution client to perform any mailer operation.
  4. After finishing the "copying and moving mail(s) to a folder" operation in the IMAP account, the evolution client immediately starts fetching the summary information for new messages. Because of this, there is a substantial delay in displaying the next mail in the list.

Calendar

Address Book